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I.  INTRODUCTION 


A.  ESTABLISHING  A  FOCUS 

1.  Background 

On  February  14,  2006,  the  Joint  Improvised  Explosive  Device  Defeat 
Organization  (JIEDDO)  was  established.  Subcomponents  of  JIEDDO  include  multiple 
boards,  one  of  which  is  the  Joint  Improvised  Explosive  Device  Defeat  Test  Board  (JTB). 
According  to  Department  of  Defense  Directive  2000. 19E,  “The  JTB  coordinates  and 
synchronizes  all  Joint  IED  Defeat  test  and  evaluation  (T&E)  events  within  the 
Department  of  Defense  and  assists  the  Military  Departments  to  maximize  utility  and 
decrease  redundancy  in  testing  of  Joint  IED  Defeat  initiatives”  (p.  16).  Due  to  the 
complexity  of  the  environment,  structure  of  the  organization,  socio-cultural  factors,  and 
lack  of  incentive  to  share  knowledge  among  test  conductors,  the  JTB  is  currently  not 
accomplishing  its  mission  as  efficiently  as  possible. 

Improvised  explosive  devices  (IEDs)  continue  to  be  the  enemy’s  weapon  of 
choice.  The  dynamic  and  responsive  enemy  alters  IEDs  in  reaction  to  our  Counter  IED 
(C-IED)  efforts,  which  results  in  the  need  for  continuous  accelerated  testing  and 
acquisition  of  C-IED  tools. 

Over  the  last  five  years  of  this  accelerated  acquisition  process,  testing  entities  and 
ranges  have  expanded  beyond  the  purview  of  the  JTB.  Personnel  perfonning  functions  in 
support  of  the  JTB  have  little  incentive  to  share  knowledge.  In  fact,  due  to  competition 
for  funding  sources,  in  some  cases,  there  is  a  disincentive  to  share  infonnation  among  test 
ranges. 

In  an  effort  to  improve  infonnation  sharing  within  the  JTB,  the  Naval 
Postgraduate  School  (NPS)  has  embarked  on  a  research  campaign  to  conduct  thesis 
research  in  order  to  help  make  the  JIEDDO  test  program  more  successful.  A  knowledge 
engineering  effort  was  conducted  with  the  goal  of  developing  an  understanding  of  the 
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critical  tasks  and  required  information  types  involved  in  the  C-IED  process.  The  final 
objective  is  to  improve  information  sharing  between  all  personnel  involved  with  the  JTB. 
This  knowledge  engineering  effort  and  a  related  end-user  analysis  are  the  first  two  in  the 
series  of  six  theses  planned  as  part  of  this  campaign.  They  may  be  followed  by  theses  on 
JTB  portal  design,  a  second  cognitive  task  analysis  on  three-dimensional  user  interfaces, 
a  C-IED  optimization  tool,  and  a  three-dimensional  JTB  portal  design. 

It  is  important  to  understand  that  there  is  a  difference  between  the  composition 
JTB,  as  defined  in  DoD  Directive  2000. 19E,  and  the  enterprise  of  organizations  and 
individuals  that  process  and  conduct  research  and  testing  to  accomplish  the  stated 
mission.  The  scope  of  this  thesis  research  was  limited  to  information  sharing  regarding 
test  execution  and  reporting  for  the  JTB.  Interviews  were  conducted  with:  (1)  individuals 
within  the  JTB;  (2)  members  of  the  advanced  communications  and  foreign  release 
working  groups;  (3)  test  directors  at  Yuma  Proving  Grounds  (YPG),  Aberdeen  Proving 
Grounds  (APG),  and  Naval  Air  Warfare  Center  Weapons  Division  China  Lake  (China 
Lake);  and  (4)  other  subject  matter  experts  with  JTB  experience.  Interview  questions 
focused  on  what  information  each  individual  needed,  expected,  received,  failed  to 
receive,  and  provided. 

This  research  was  required  because  the  lack  of  information  sharing  between  these 
entities  contributed  to  duplication  of  effort,  inefficient  use  of  resources,  and  most 
importantly,  delays  in  fielding  equipment  to  the  war  fighter  (U.S.  Government 
Accountability  Office,  2009).  Typically,  the  JTB  would  send  requirements, 
prioritization,  coordination,  and  resources,  and  receive  analysis,  and  reporting  from 
working  groups,  program  managers  (PMs),  and  Joint  Program  Executive  Offices.  As  of 
the  writing  of  this  thesis,  information  did  not  flow  as  needed  for  all  personnel  associated 
with  testing  C-IED  weapon  systems. 

2.  Research  Questions 

In  the  conduct  of  this  thesis,  the  researcher  focused  on  two  questions.  What 
impediments  exist  to  information  sharing  within  the  JTB?  How  can  those  impediments 
be  mitigated?  Multiple  secondary  supporting  questions  were  also  considered. 
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II.  BACKGROUND 


A.  CONTEXT 

The  objective  of  this  thesis  was  to  improve  information  sharing  in  the  JTB 
organization;  the  method  to  achieve  that  objective  was  a  cognitive  engineering  analysis. 
It  describes  how  users  decide  what  information  to  share  and  with  whom.  Before 
considering  their  decision-making  process,  however,  one  must  have  an  understanding  of 
the  concepts  that  will  be  discussed  in  the  following  chapters.  This  chapter  provides  a 
review  of  information  sharing,  organizational  considerations,  and  cognitive  systems 
engineering. 

B.  INFORMATION  SHARING:  WHAT  IT  IS  AND  WHY  PEOPLE  DO  IT 

1.  What  It  Is 

Although  information  sharing  seems  like  a  straightforward  concept,  it  has 
different  connotations  that  depend  on  the  context  of  the  discussion.  For  example,  in 
politics  it  often  refers  to  interagency  coordination,  while  in  the  medical  field  it  refers  to 
privacy  of  individual  medical  records.  Because  this  project  is  focused  within  the 
Department  of  Defense  (DoD),  the  definition  and  understanding  of  information  sharing 
will  be  based  on  the  Department  of  Defense  Information  Sharing  Strategy  (DoD 
Strategy).  The  DoD  strategy  is  based  on  the  National  Strategy  for  Information  Sharing 
(National  Strategy)  and  the  2006  Quadrennial  Defense  Review  (QDR);  it  is  implemented 
through  the  DoD  Infonnation  Sharing  Implementation  Plan.  The  National  Strategy  is 
focused  on  combating  terrorism.  The  vision  is  stated,  in  part,  to  “establish  a  more  inte¬ 
grated  infonnation  sharing  capability  to  ensure  that  those  who  need  information  to  protect 
our  Nation  from  terrorism  will  receive  it  and  those  who  have  that  information  will  share 
it”  (National  Security  Council  [NSC],  2007,  p.  2).  It  does  an  excellent  job  of  providing 
guidance  to  that  end;  however,  the  closest  that  it  comes  to  defining  the  concept  is,  “The 
exchange  of  infonnation  should  be  the  rule,  not  the  exception,  in  our  efforts  to  combat 
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the  terrorist  threat.”  (NSC,  2007,  p.  1)  What  makes  it  relevant  to  this  thesis  is  that  it 
influenced  the  creation  of  the  DoD  Strategy,  which  broadens  the  vision  and  provides  a 
working  definition  of  the  concept.  The  DoD  strategy  was  published  in  response  to  the 
QDR  decision  to,  “Strengthen  its  data  strategy  ...  to  improve  information  sharing  and 
information  assurance,  and  extend  it  across  a  multitude  of  domains  .  .  .  [and]  Develop  an 
information- sharing  strategy  to  guide  operations  with  Federal,  state,  local  and  coalition 
partners.”  (U.S.  Department  of  Defense  [DoD],  2006,  p.  59). 

The  DoD  Strategy  broadened  the  vision  from  terrorism  to  one  that  is  applicable 
throughout  its  area  of  influence;  the  stated  vision  is,  “Deliver  the  power  of  information 
to  ensure  mission  success  th  rough  an  agile  enterprise  with  freedo  m  of 
maneuverability  across  the  information  environment”  (U.S.  Department  of  Defense 
[DoD],  2007,  p.  3).  (Bold  text  and  underlining  are  included  in  the  original.)  The  DoD 
Strategy  definition  of  information  sharing,  which  will  be  the  standard  throughout  this 
thesis,  is  ‘“Making  information  available  to  participants  (people,  processes,  or 
systems).’  Information  sharing  includes  the  cultural,  managerial,  and  technical  behaviors 
by  which  one  participant  leverages  information  held  or  created  by  another  participant” 
(DoD,  2007,  p.  ii).  (The  quotation  marks  and  bold  text  are  included  in  the  original 
document.)  It  is  significant  to  note  that  the  definition  specifies  infonnation  sharing  as  a 
behavior,  which  means  that  it  cannot  be  relegated  to  a  task  of  an  infonnation  system.  The 
Strategy  includes  appropriate  changes  to  the  culture  and  approaches  to  implement,  but  it 
does  not  address  what  motivates  people  to  share. 

2.  Why  People  Do  It 

The  culture  will  not  change,  nor  will  the  approaches  be  successful  if  individuals 
are  not  motivated  to  share  and  if  they  do  decide  to  share,  there  is  a  difference  between 
sharing  information  and  sharing  tacit  knowledge.  Research  by  Lam  and  Lambermont- 
Ford  (2010)  indicates  there  are  three  categories  of  motivation  to  share  infonnation: 
intrinsic  normative,  intrinsic  hedonic,  and  extrinsic.  Intrinsic  normative  motivation  is 
sharing  because  one  feels  that  it  is  the  right  thing  to  do.  In  the  case  of  the  research  topic 
for  this  thesis,  a  member  of  the  JTB  organization  who  is  motivated  by  intrinsic  nonnative 
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motivation  would  share  information  because  the  person  would  consider  it  a  job 
requirement.  Intrinsic  hedonic  motivation  involves  sharing  information  because  it  makes 
one  feel  good.  In  this  example,  the  member  feeling  intrinsic  hedonic  motivation  would 
share  information  because  it  would  make  the  person  feel  good  that  war  fighters  may 
benefit  from  it.  Extrinsic  motivation  is  sharing  information  in  return  for  something  of 
value.  The  member  in  this  case  would  share  information  if  it  would  result  in  a 
promotion.  What  makes  motivating  an  enterprise  to  improve  information  sharing  is  that 
different  members  are  motivated  by  different  permutations  of  the  categories  and  the 
pennutation  changes  for  each  member  changes  throughout  the  day. 

To  further  complicate  the  situation,  the  DoD  definition  specifies  sharing  of 
information  held  or  created  by  another  participant.  Information  held  by  another  can  be 
considered  tacit  knowledge  or  an  information  product.  Tacit  knowledge,  similar  to 
expertise,  is  understood  without  being  part  of  a  document.  Information  products,  on  the 
other  hand,  are  tangible  documents.  People  view  the  sharing  of  these  two  types  of 
information  differently.  Essentially,  the  permutations  of  the  motivational  categories  are 
different,  so  one  may  share  an  information  product  because  of  intrinsic  normative 
motivation,  but  require  intrinsic  hedonic  motivation  to  share  tacit  knowledge.  Another 
difference  between  these  types  of  information  is  the  medium  used  for  sharing. 
Information  products  are  easily  shared  via  infonnation  systems.  The  unwritten  nature  of 
tacit  knowledge,  on  the  other  hand,  makes  it  difficult  to  document  and  disseminate. 
According  to  research  by  Constant,  Kiesler,  and  Sproul  (1994)  and  Holste  and  Fields 
(2010),  tacit  knowledge  sharing  also  varies  with  the  level  of  trust  the  individual  has  with 
his  coworkers  more  than  sharing  infonnation  products.  The  higher  the  level  of  trust,  the 
more  tacit  knowledge  is  shared. 

C.  ORGANIZATIONAL  CONSIDERATIONS:  HOW  WE  ENCOURAGE  IT 

1.  Organizational  Culture 

Considering  the  complexity  of  different  motivations  for  sharing  information,  an 
organization  must  create  a  diverse  culture  in  order  to  stimulate  as  many  members  as 
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possible  to  participate.  The  individual  stimulation  must  be  generated  through  an 
organizational  culture  that  is  strong  enough  and  focused  enough  to  influence  a  member’s 
mindset.  Although  it  is  true  that  “successful  information  sharing  necessitates  a  mindset 
where  information  is  continually  shared  as  a  normal  course  of  work,”  (DoD,  2007,  p.  5) 
establishing  that  mindset  requires  a  strong  supportive  organizational  culture.  There  are 
many  definitions  of  organizational  culture,  and  most  are  founded  on  an  organization’s 
values,  beliefs,  and  norms.  For  the  purpose  of  this  thesis,  the  value  considered  is  the 
sharing  of  information.  Beliefs  and  norms  will  be  addressed  through  policy  and  process, 
respectively. 

Beliefs  for  any  organization  must  be  established  by  an  authority  figure  and 
disseminated  to  the  members.  Because  it  originates  from  an  authority  figure,  members 
accept  it  as  legitimate  and  a  single  authoritative  source  provides  the  common  description 
of  the  belief.  To  put  it  plainly,  “to  encourage  sharing  of  work  information,  organizations 
need  a  clear  policy.”  (Constant  et  ah,  1994,  p.  418)  As  the  initiation  of  the 
implementation  of  the  value  to  share  information,  a  policy  must  not  only  exist,  it  must 
provide  “clear,  concise,  and  comprehensive  guidance”  (DoD,  2007,  p.  1 1).  Depending  on 
the  organizational  structure,  multiple  policies  may  be  required  at  different  levels  or 
knowledge  hubs.  It  is  essential  that  all  policies  are  synchronized  to  avoid  conflicting 
guidance  (DoD,  2007). 

The  policy  must  direct  members  to  take  specific  standardized  actions,  or  norms, 
though  the  processes  that  it  directs.  While  the  most  important  aspect  of  information 
sharing  is  motivating  the  individuals  to  participate,  the  organizational  leadership  must 
provide  a  standard  process  or  set  of  processes  to  enable  the  members  to  take  action  and 
streamline  evaluation.  As  a  result,  they  will  be  able  to  take  uniform  action  toward  a 
common  goal.  In  order  to  be  effective,  the  processes  must  be  created  with  the  user  in 
mind,  assigned,  enforced,  and  reviewed  for  barriers  to  their  purpose. 

When  creating  the  new  processes,  relating  them  to  existing  procedures  or  tasks 
will  ease  the  transition  to  the  new  process.  It  will  also  speed  the  cultural  assimilation  or 
“user  buy-in”  that  is  necessary  to  be  effective  (McLaughlin,  2010). 
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Just  as  the  policy  must  be  clear  and  easy  to  understand,  the  processes  it  directs 
must  be  assigned  to  specific  member  positions  for  responsibility  of  compliance  and 
communication  (McLaughlin,  2010).  Ideally,  because  the  process  is  related  to  current 
practice,  compliance  should  not  be  much  of  an  issue.  However,  the  assigned  individual 
must  have  the  authority  and  ability  to  enforce  compliance  when  necessary.  The 
responsible  member  must  also  serve  as  the  point  of  contact  for  clarification  and 
suggestions  for  improvement.  Even  in  the  ideal  situation  presented  previously, 
enforcement  is  required. 

Whether  members  lack  the  motivation  to  share,  make  errors,  or  deliberately 
intend  to  neglect  the  policy,  the  responsible  member  will  need  to  enforce  compliance 
with  the  process.  The  manner  of  enforcement  will  depend  on  the  nature  of  the  policy  and 
procedure,  lack  of  compliance,  organizational  structure,  and  multiple  other  variables  to  be 
deliberated  by  the  responsible  member. 

Following  implementation,  the  processes  must  also  be  reviewed  at  regular 
intervals  in  order  to  ensure  that  they  are  still  applicable  and  identify  barriers  to 
compliance  (McLaughlin,  2010).  As  organizations  evolve,  their  processes  must  change  to 
maintain  applicability  to  the  spirit  and/or  letter  of  the  policy  and  evoke  the  organizational 
culture.  Considering  the  nature  of  interaction  required  for  infonnation  sharing,  if  one 
element  of  the  organization  encounters  a  barrier  and  fails  to  participate,  the  entire 
organization  is  affected.  By  identifying  barriers  to  compliance,  they  can  be  addressed 
and  mitigated  to  the  greatest  possible  extent.  Identifying  the  barriers  to  sharing  and 
recommending  ways  to  mitigate  them  is  the  central  focus  of  this  thesis. 

2.  Organizational  Environment 

Once  the  organizational  culture  has  been  established  and  conditions  for 
motivating  members  to  share  infonnation  have  been  developed,  the  tools  for  members  to 
conduct  the  act  of  sharing  must  be  implemented.  Social  interaction,  incentives,  and 
information  systems  are  the  tools  that  create  the  organizational  environment. 

The  social  interaction  aspect  of  the  organizational  environment  is  the  most  direct 

means  to  motivate  information  sharing.  Methods  of  implementation  include  exploiting 
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social  networks,  increasing  interaction,  and  increasing  collaboration.  Exploiting  social 
networks  essentially  puts  the  power  of  peer  pressure  to  work.  By  studying  the  social 
networks  of  an  organization,  it  is  possible  to  identify  members  who  act  as  sharing  hubs 
within  their  network,  bridges  to  other  social  networks,  and  links  to  providers  (Sun,  2010). 
Exploiting  members  in  those  unofficial  positions  will  create  positive  effects  because 
people  will  be  comfortable  in  their  roles  and  fewer  new  relationships  will  have  to  be 
established  than  if  social  networks  were  ignored. 

Something  as  simple  as  including  greater  social  interaction  can  improve  the 
environment.  Ironically,  as  individual  communication  technology  continues  to  evolve, 
people  spend  less  time  communicating  face  to  face.  In  light  of  that  trend,  research  has 
found  that  “the  best  organizational  policy  may  be  simply  to  create  occasions  for  people  to 
talk  and  exchange  knowledge,  opinions,  and  advice”  (Constant  et  ah,  1994,  p.  419).  The 
organizational  environment  may  be  improved,  then,  by  having  more  meetings,  training 
members  on  how  to  conduct  meetings  more  efficiently,  and/or  increasing  participation  in 
large  professional  gatherings  like  conventions  and  symposiums.  Sharing  tacit  knowledge 
may  also  increase  because  McAllister’s  research  (Holste  &  Fields,  2010)  has  shown  that 
“higher  levels  of  affiliative  behavior  (taking  a  personal  interest,  passing  on  infonnation, 
helping  another  with  tasks)  increased  affect-based  trust  among  professional  peers” 
(Holste  &  Fields,  2010,  p.  136). 

Increasing  collaboration  among  members  improves  the  organizational 
environment  by  increasing  trust  and  incorporating  social  interaction.  The  ability  of  an 
organization  to  collaborate  on  a  large  scale  requires  a  well-established  operational 
environment,  but  benefits  can  also  be  obtained  through  small  scale  collaboration 
exercises.  The  benefits  are  greatly  based  on  trust,  which,  according  to  McAllister  and 
Dietz  (Holste  &  Fields,  2010),  “may  be  increased  through  frequent  direct  engagement  of 
co-workers  in  collaborative  processes — especially  situations  that  illustrate 
interdependency  and  provide  opportunity  for  workers  to  demonstrate  individual 
competency  (Holste  &  Fields,  2010,  p.  136).  The  aspect  of  direct  engagement  requires 


8 


social  interaction.  The  benefits  of  physical  social  interaction  have  been  presented,  but  the 
research  did  not  put  a  physical  restraint  on  direct  engagement.  Information  systems  could 
be  incorporated  into  the  collaborative  process  as  well. 

Many  organizations  decide  to  include  incentives  in  the  environment  to 
extrinsically  motivate  members  to  share  information.  In  order  to  be  effective,  the 
incentives  must  fit  the  type  of  infonnation  to  be  shared  and  the  tools  available  (Hansen, 
Nohria,  &  Tierney,  1999;  Sun,  2010).  For  example,  if  the  goal  of  the  organization  is  for 
the  members  to  share  more  information  products,  the  incentive  should  be  related  to  the 
number  of  products  created  or  entered  into  an  infonnation  system.  If  the  goal  is  to 
increase  the  sharing  of  tacit  knowledge,  incentivizing  document  creation  would  be 
ineffective.  Rather,  the  incentive  should  focus  on  productive  social  interaction  which 
results  in  the  passing  on  of  tacit  knowledge. 

Infonnation  systems  are  often  seen  as  the  silver  bullet  for  solving  information 
sharing  issues.  Although  they  are  a  significant  aspect  of  the  operational  environment  and 
greatly  enhance  an  organization’s  ability  to  share  information,  they  are  only  as  good  as 
the  members  who  operate  them.  When  considering  including  an  information  system  in 
an  operational  environment,  the  planners  must  consider  whether  to  use  a  codification  or 
personalization  strategy.  When  studied  from  the  proper  perspective,  as  tools  that 
members  use  to  share  information,  when  appropriately  incorporated  into  the  strategy, 
their  contribution  falls  into  one  of  two  categories:  social  interaction  or  data  manipulation. 

While  all  information  sharing  strategies  are  unique,  they  can  be  categorized  as  a 
codification  or  personalization  strategy.  In  codification,  information  is  “extracted  from 
the  person  who  developed  it,  made  independent  of  that  person,  and  reused  for  various 
purposes”  (Hansen  et  ah,  1999,  p.  108).  An  information  system  with  a  large  repository  to 
mine  for  information  would  be  appropriate  for  this  type  of  strategy.  In  personalization, 
on  the  other  hand,  the  information  is  largely  tacit,  so  a  large  repository  would  not  be 
appropriate.  An  information  system  for  that  strategy  would  be  more  effective  if  it  were 
based  on  interpersonal  communication  and  included  tools  for  collaboration.  Of  course  all 
organizations  will  have  some  aspects  of  each  category  in  their  strategy,  but  only  one 

should  be  followed  to  ensure  unity  of  effort  (Hansen  et  ah,  1999). 
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The  reason  for  and  benefits  of  social  interaction  have  been  covered,  but  it  is  not 
feasible  to  bring  everyone  together  or  send  them  to  conferences.  Social  interaction  can 
be  achieved  through  an  information  system  with  similar  effects  to  physical  interaction. 
The  systems  can  be  particularly  successful  if  they  incorporate  a  method  to  express  the 
need  for  the  infonnation  request  and  feedback  to  the  expert  that  provides  the  answer 
(Constant  et  ah,  1994). 

Data  manipulation  is  not  different  for  infonnation  systems  designed  for 
information  sharing  than  those  used  for  any  other  reason.  However,  the  ability  to 
disseminate,  store,  send,  receive,  and  process  data  regardless  of  physical  location  is  an 
essential  element  of  the  infonnation  sharing  environment. 

3.  Organizational  Structure 

The  structure  of  an  organization  establishes  the  relationships  and  authorities  of 
suborganizations.  In  doing  so,  it  impacts  all  other  elements  of  information  sharing 
between  personnel  who  are  part  of  the  organization.  However,  because  the 
organizational  structure  is  influenced  by  many  other  factors,  even  if  information  sharing 
is  a  high  priority,  the  strategy  must  be  flexible  for  proper  integration.  The  organizational 
structure  can  create  barriers  to  infonnation  sharing,  which  must  be  recognized,  mitigated, 
and  reviewed  as  it  evolves  (McLaughlin,  2010).  Therefore,  planners  must  understand 
both  hierarchical  and  nonhierarchical  organizational  structures  and  how  information 
typically  flows  within  them. 

Hierarchical  organizations  separate  areas  of  expertise  in  order  to  process 
information  prior  to  disseminating  it  to  other  areas  of  expertise  as  input  or  informing  a 
decision  maker,  see  Figure  1.  Processing  by  areas  of  expertise  ensures  accuracy  and 
streamlines  infonnation  tracking.  Sharing  information  between  suborganizations  with 
different  areas  of  expertise  is  relatively  difficult  due  to  both  culture  and  communication 
equipment  connectivity. 
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Figure  1.  Example  of  a  Hierarchical  Organization 

Nonhierarchical  organizations  integrate  areas  of  expertise  into  work  teams  in 
order  to  “decentralise  authority,  share  information,  diffuse  and  distribute  competency” 
(Klenke,  2006,  p.  231),  see  Figure  2.  Integration  of  expertise  fosters  “innovation,  product 
customisation,  and  technological  leverage”  (Klenke,  2006,  p.  231).  Ensuring  thorough 
dissemination  of  corroborated  accurate  information  to  all  work  teams  and  consensus 
building  are  challenges  in  this  structure  due  to  both  culture  and  communication 
equipment  connectivity.  Trust  is  more  significant  in  nonhierarchical  organizations  in 
order  for  consensus  building  to  be  effective. 
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Figure  2.  Example  of  a  Nonhierarchical  Organization 


D.  COGNITIVE  SYSTEMS  ENGINEERING:  HOW  TO  FACILITATE  IT 

1.  Introduction  to  Cognitive  Systems  Engineering 

Cognitive  systems  engineering  is  a  process  used  to  design  computer  systems 
based  on  integrating  “the  fields  of  human  factors,  human-computer  interaction,  cognitive 
psychology,  computer  science,  artificial  intelligence  and  other  related  fields” 
(Introduction,  n.d.,  para.  2).  One  of  the  approaches  used  for  cognitive  systems 
engineering  is  participatory  design,  which  “places  a  premium  on  the  active  involvement 
of  workplace  practitioners. ..in  design  and  decision-making  processes”  (Participatory 
Design,  n.d.,  para.  1).  The  participatory  design  methods  of  cognitive  systems  engineering 
integrated  for  this  thesis  are  critical  decision  making  and  contextual  inquiry. 

2.  Critical  Decision  Method 

The  critical  decision  method  is  one  of  several  methods  used  to  conduct  a 
cognitive  task  analysis  (CTA).  A  CTA  is  used  to  “analyze  and  represent  the  knowledge 
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and  cognitive  activities  workers  utilize  to  perform  complex  tasks  in  a  work  domain” 
(Gordon  &  Gill,  1997;  Schraagen,  Chipman,  &  Shalin,  2000  in  Bonaceto  &  Bums,  2007, 
p.  40).  Critical  decision  making  in  particular  is  a  “structured  interview  technique 
developed  to  obtain  infonnation  about  decisions  made  by  practitioners  when  performing 
their  tasks”  (Hutchins,  Pirolli,  &  Card,  2007,  p.  290).  Essentially,  the  method  elicits  a 
detailed  response  from  the  interviewee  by  asking  him  to  address  an  extreme  atypical 
situation.  By  listening  to  the  description  and  asking  additional  probing  questions,  the 
interviewer  attempts  to  tease  out  the  essential  elements  of  the  decision  making  process. 
The  decision-making  processes  of  the  interviewees  are  analyzed  and  used  to  provide 
input  to  the  systems  engineering  process. 

3.  Contextual  Inquiry 

The  goal  of  contextual  inquiry  is  to  “support,  extend,  and  positively  transform  the 
work  of  individuals,  teams,  and  businesses  through  computer  systems”  (Holtzblatt  & 
Jones,  1993,  p.  177).  To  reach  that  goal,  it  attempts  to  identify  a  process  for  designing 
systems  that  support  people  who  engage  in  similar  work  in  many  different  business 
contexts  and  cultures  ...  an  expedient  process  for  gathering  user  infonnation  in  the  time 
available  ...  a  means  of  gaining  appropriate  and  helpful  information  about  users’  work. 
(Holtzblatt  &  Jones,  1993,  p.  177) 

The  contextual  inquiry  method  is  particularly  useful  because  of  its  structure.  It 
provides  a  utilitarian  balance  of  theory  and  practice.  It  was  conceived  to  be  used  with 
other  fonns  of  design.  The  formative  principles  of  context,  partnership,  and  focus, 
“guide  our  interaction  with  users  regardless  of  the  specific  technique  or  meeting  structure 
that  we  employ”  (Holtzblatt  &  Jones,  1993,  p.  179).  At  the  same  time,  it  provides  the 
researcher  with  detailed  steps  to  put  the  principles  into  practice  through  interviews  and 
analysis. 

In  contrast  to  the  structured  interviews  of  the  CTA,  contextual  inquiry  interviews 
occur  while  observing  and  interacting  with  interviewees  while  they  are  conducting 
routine  tasks.  Individual  or  group  interviewers  monitor  interviewees  and  engage  them  in 
conversation  throughout  the  interview.  The  intent  is  to  observe  how  the  interviewees  are 
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using  the  tools  provided,  record  their  input,  and  identify  barriers  that  the  users  themselves 
overlook.  Rather  than  having  a  list  of  structured  interview  questions,  the  interviewers 
arrive  with  a  well  thought  out  focus  of  the  interview.  As  the  interview  progresses,  the 
interviewers  adjust  their  focus  to  interpret  the  most  realistic  view  possible  of  the  working 
environment.  During  the  wrap  up  of  the  interview,  the  interviewers  confirm  that  their 
interpretation  is  accurate  (Holtzblatt  &  Jones,  1993).  Multiple  rounds  of  interviews  are 
often  conducted. 

Analysis  of  the  interviews  is  best  done  by  a  group  but  can  be  done  by  an 
individual  when  appropriate.  The  product  created  through  contextual  inquiry  analysis  is 
the  affinity  diagram.  The  diagramming  process  “is  a  way  to  synthesize  qualitative 
information  into  conceptual  groupings”  (Holtzblatt  &  Jones,  1993,  p.  203).  It  starts  by 
refining  the  focus  for  analysis  and  reviewing  the  interview  transcripts  in  that  light. 
During  the  review  process,  significant  aspects  are  recorded  in  detail  and  capsulated  on 
note  paper.  After  reviewing  all  transcripts,  the  pieces  of  note  paper  are  grouped  by  how 
they  relate  to  each  other.  Analysts  shift  note  papers  between  groups  until  they  are 
satisfied  that  the  best  relationships  are  represented.  They  then  name  the  groups,  which 
become  level  one  of  the  affinity  diagram.  The  analysts  group  and  name  the  groups  two 
more  times,  resulting  in  a  three-level  hierarchy  of  concepts  to  guide  the  engineering  of 
the  system  (Holtzblatt  &  Jones,  1993). 

E.  SUMMARY 

1.  Context 

The  goal  of  this  thesis  is  to  provide  an  analysis  of  infonnation  sharing  within  the 
JTB,  which  will  provide  information  to  inform  researchers  regarding  JTB  portal  design 
requirements  for  the  follow  on  theses  in  the  series.  In  order  to  analyze  information 
sharing,  one  must  study  the  meaning  of  information  sharing,  organizational 
considerations,  and  relevant  methods  of  cognitive  systems  engineering  that  link  the 
human  and  technological  aspects.  Studying  the  meaning  of  infonnation  sharing  requires 
a  common  definition  and  understanding  of  individual  motivation  to  share.  Organizational 
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considerations  consist  of  the  organizational  culture,  environment,  and  structure.  The 
methods  of  cognitive  systems  engineering  that  are  relevant  to  this  thesis  are  CTA  and 
contextual  inquiry.  The  way  these  methods  were  employed  will  be  covered  in  the 
following  chapter. 
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III.  METHODOLOGY 


A.  RESEARCH  METHODOLOGY 

1.  Commencement 

The  kickoff  meeting  for  this  project  provided  focus,  delineated  the  scope,  and 
recommended  the  initial  research  method.  The  thesis  advisors  were  able  to  focus  the 
student  researchers  on  the  mission  of  this  series  of  theses  by  providing  an  introduction  to 
the  JTB  organization,  its  purpose,  the  current  testing  process,  and  the  planned 
contribution  of  the  project. 

This  introduction  necessarily  included  organizations  other  than  the  JTB  in  order 
to  provide  a  comprehensive  framework  that  described  the  entire  JIEDDO  enterprise  for 
the  researchers.  Follow-on  discussions,  however,  limited  the  scope  for  each  of  the  three 
initial  theses.  Limiting  the  scope  was  important  because  it  deconflicted  the  researchers, 
thus  enabling  coordination.  The  scope  for  this  thesis  was  limited  to  information  sharing 
regarding  test  execution  and  reporting  for  the  JTB.  Interviewees  included  (1)  individuals 
within  the  JTB;  (2)  members  of  the  advanced  communications  and  foreign  release 
working  groups;  and  (3)  test  directors  at  YPG,  APG,  and  China  Lake.  Interview 
questions  focused  on  what  information  each  individual  needs,  expects,  receives,  fails  to 
receive,  and  provides. 

This  researcher  was  directed  to  use  the  critical  decision  method  of  cognitive  task 
analysis.  In  accordance  with  the  directive,  structured  interviews  were  required.  Due  to 
the  similar  scope  of  the  first  two  theses  in  the  project  and  practical  scheduling  and 
logistical  requirements,  the  researchers  planned  to  conduct  joint  interviews.  This 
decision  was  significant  because  it  positively  affected  the  researchers  by  enabling  support 
during,  coordination  between,  and  joint  review  of  the  interviews. 
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B. 


PREPARATION 


1.  Concurrent  Processes 

The  process  of  preparing  for  data  collection  consisted  of  the  compilation  of  an 
Institutional  Review  Board  package,  selection  of  participants,  and  rehearsals.  Planning  of 
all  aspects  was  conducted  concurrently  by  the  researchers,  advisors,  and  sponsor 
representatives  in  order  to  streamline  the  transition  from  preparation  to  collection. 

2.  Participant  Selection 

Selection  of  participants  for  data  collection  for  both  this  thesis  and  the  end-user 
analysis  was  a  coordinated  effort  between  the  students,  advisors,  and  sponsor 
representative.  The  advisors  coordinated  the  interviews  at  YPG  and  Naval  Air  Warfare 
Center  Weapons  Division  Point  Mugu  (Point  Mugu),  provided  some  recommendations 
for  other  locations,  and  put  the  students  in  touch  with  the  sponsor.  Student  researchers 
coordinated  the  interview  at  China  Lake  and  provided  input  to  the  sponsor  regarding  the 
focus  of  their  topics.  The  sponsor  coordinated  the  interviews  at  APG  and  the  JTB  in 
Alexandria,  Virginia. 

Participants  were  selected  based  on  both  their  knowledge  and  experience  with 
JTB  testing  and  availability  during  the  students’  limited  travel  availability  due  to  class 
schedules.  The  plan  was  to  include  participants  with  knowledge  and  experience  from 
different  perspectives  of  the  testing  process.  Perspectives  of  the  testers/processors, 
support  personnel,  and  facilitators  were  all  relevant  and  successfully  targeted  in  the 
selection  process.  The  plan  was  successfully  executed  by  interviewing.  Student  travel 
windows  were  rigid  because  of  travel  planning  considerations  and  concurrent  class 
schedules.  The  schedule  rigidity  resulted  in  the  students’  inability  to  interview  the  Acting 
Director  and  Deputy  Director,  due  to  a  scheduling  conflict. 

3.  Rehearsals 

If  one  is  replicating  this  process,  it  is  prudent  at  this  point  to  acknowledge  the 


benefit  of  realistic  rehearsals.  This  researcher  rehearsed  by  reading  through  the  questions 
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aloud  in  order  to  familiarize  himself  with  the  flow  and  timing  of  the  questions.  Such 
rehearsal  prepared  him  to  present  the  questions  in  a  professional  manner;  it  did  not 
prepare  him  for  handling  unfocused  or  misdirected  answers.  This  resulted  in  a  lack  of 
standardization  in  analysis  because  some  questions  were  omitted  and  others  were 
answered  from  a  perspective  irrelevant  to  the  focus  of  the  project.  For  follow  on 
researchers,  regardless  of  the  amount  of  time  available  before  collection,  at  least  one  live, 
recorded,  and  timed  rehearsal  interview  with  a  prepared  interviewee  is  recommended. 
Even  if  the  interviewee  is  not  an  expert  in  the  field,  the  benefits  of  operationally  checking 
all  equipment,  taking  notes,  addressing  interviewee  questions,  and  facilitating  the  focus 
of  the  interview,  will  improve  confidence  in  oneself  and  one’s  questions,  the  conduct  of 
the  actual  interviews,  and  the  standardization  of  the  process  for  more  efficient  analysis. 

C.  INITIAL  DATA  COLLECTION 
1.  Interviews  and  Surveys 

a.  Overview 

The  initial  data  collection  phase  consisted  of  1 1  structured  interviews, 
demographic  surveys,  and  information  sharing  surveys.  The  interviews  and  surveys  had 
the  same  target  audience,  addressed  in  Chapter  III. A.  1 .  Both  surveys  were  given  to  the 
interviewees  and  distributed  and  collected  during  the  interview  session.  Locations  and 
number  of  interviews  conducted  at  each  location  were:  YPG  (3),  China  Lake  (1),  APG 
(1),  JTB  (4),  Point  Mugu  (1),  and  NPS  (1).  All  of  the  student  researcher  interviews  were 
conducted  jointly  with  the  end-user  analysis  researcher. 

There  were  two  overarching  factors  that  affected  the  interviews.  First, 
some  questions  were  omitted,  at  times  purposefully,  and  at  other  times,  erroneously. 
They  were  purposely  omitted,  if  they  were  already  answered  or  clearly  not  applicable  to 
the  interviewee.  Others,  however,  were  omitted  erroneously  by  this  interviewer.  The 
second  issue  was  that  two  of  the  interviews  were  not  recorded.  One  interviewee  did  not 
consent,  which  was  acceptable.  The  other  was  not  recorded  because  the  interview  was 
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conducted  in  a  secure  location  where  recorders  were  not  allowed.  The  lack  of  a  transcript 
greatly  reduced  the  amount  of  data  collected  from  that  interview.  Future  researchers  are 
advised  to  ensure  that  interviews  are  conducted  in  an  environment  where  recording  is 
allowed,  in  order  to  have  that  option,  should  the  interviewee  consent. 

(1)  Interviews.  The  interview  fonnat  separated  the  questions  into 
five  sections:  introduction,  orientation,  information  sharing,  information  flow  policy  and 
procedure,  and  conclusion.  The  text  of  the  questions  can  be  found  in  tables  in  the  results 
chapter.  Only  the  data  collected  in  the  information  sharing  and  information  flow  policy 
and  procedure  sections  was  used  for  analysis.  The  questions  addressed  organizational 
culture,  environment,  and  structure,  as  well  as  the  aspect  of  participatory  design  in 
cognitive  engineering. 

The  information  sharing  section  of  the  overall  set  of  interview 
questions  consisted  of  16  questions  that  assessed  the  state  of  infonnation  sharing  within 
the  organization,  identified  barriers,  and  solicited  recommendations  for  improvement. 
Organizational  culture  norms  were  addressed  in  Questions  5-11  and  14.  The  intent  of 
these  questions  was  to  document  the  existing  norms  for  information  sharing  at  multiple 
levels  within  the  organization.  The  utility  of  current  infonnation  systems  was  addressed 
by  inquiring  about  the  media  used  for  communication  in  Questions  6,  8,  and  11. 
Interviewee  recommendations  solicited  in  Questions  12,  13,  15,  and  16  provided 
participant  data  for  future  design  considerations.  Questions  1-4  did  not  address  specific 
motivational  or  engineering  aspects,  however,  in  establishing  a  dialogue  and  eliciting  a 
critical  decision  making  situation,  they  provided  an  avenue  for  the  interviewee  to  address 
the  aspects  most  significant  to  him. 

The  information  flow  policy  and  procedure  section  consisted  of 
seven  questions  that  addressed  leaders’  guidance  for  sharing  information,  how 
information  was  meant  to  flow,  and  how  those  were  reflected  in  practice.  In  Question  6, 
the  normative  aspect  of  organizational  culture  was  addressed  using  information  flow;  it 
did  not  specifically  address  procedures,  which  enabled  the  interviewee  to  express  the 
actual  execution  of  moving  information  through  the  organization.  Policy  is  an  element  of 
belief  within  organizational  culture.  Question  4  addressed  both  the  existence  and 
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enforcement  of  an  information  sharing  policy  within  the  JTB.  Question  1  addressed  the 
interviewee’s  perception  of  the  organizational  structure  relevant  to  information  flow. 
Participatory  design  aspects  were  addressed  in  Questions  2,  3,  5,  and  7. 

(2)  Surveys.  The  same  demographic  and  information  sharing 
surveys  were  given  to  all  interviewees.  The  demographic  survey  addressed  individual 
characteristics,  education,  experience,  and  placement  within  the  organization. 

The  infonnation  sharing  survey  consisted  of  25  questions  broken 
into  three  sections:  organizational  infonnation  sharing  (12),  intra-organization 
communication  structure  (5),  and  organizational  culture  (8).  The  infonnation  sharing 
section  focused  on  establishing  the  norms  for  the  organizations  and  motivation.  All  of 
the  questions  except  number  two  addressed  organizational  norms  at  the  personal  level. 
The  questions  were  posed  from  different  perspectives  in  order  to  assess  not  only  the 
quantity  and  quality  of  sharing,  but  the  balance  of  giving  and  receiving  as  well.  Question 
2  addressed  motivation.  The  relatively  small  amount  of  attention  paid  to  motivation  was 
intentional.  The  focus  in  the  initial  phase  was  to  collect  data  on  what  was  or  was  not 
being  shared,  with  a  lesser  focus  on  how.  The  why  aspect,  motivation,  would  be 
addressed  in  the  follow-up  phase  if  the  initial  analysis  found  a  requirement  for  it. 

The  intra-organizational  communication  section  also  focused  on 
organizational  norms,  except  for  a  single  question  on  policy.  This  section  differed  from 
the  infonnation  sharing  section  by  taking  a  slightly  broader  perspective  and  addressing 
organizational  norms  within  the  interviewee’s  suborganization  within  the  JTB.  They 
assessed  the  quality,  requirement,  procedure  and  process  of  the  current  state  of 
information  sharing.  The  policy,  or  leadership  encouragement,  was  addressed  in  question 
number  four. 

All  questions  in  the  organizational  culture  section  addressed  norms 
from  the  broadest  perspective  within  the  scope  of  this  thesis  by  focusing  on  information 
sharing  between  the  JTB  suborganizations 
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b.  Initial  Data  Collection  Summary 

The  initial  round  of  data  collection  consisted  of  12  interviews  at  five 
organizations,  as  detailed  in  Table  1.  The  general  flow  of  the  interviews  included 
introductions,  research  overview,  infonnation  sharing  interview  questions,  end-user 
analysis  interview  questions,  and  demographic  and  information  sharing  survey 
completion.  Not  all  interviewees  were  asked  all  of  the  information  sharing  interview 
questions,  as  annotated  in  the  tables  presented  in  the  results  chapter. 


Table  1.  Initial  Interviews  by  Organization 


Organization 

Number  of  Interviews 

Advanced  Communications  Working  Group 

1 

Joint  Improvised  Explosive  Device  Defeat 
Test  Board 

5 

Yuma  Proving  Grounds 

3 

NAWCWD  China  Lake 

1 

NAWCWD  Point  Mugu 

1 

D.  INITIAL  DATA  ANALYSIS 

1,  Overview 

This  section  will  address  the  process  of  analyzing  the  initial  data  and  how  it  was 
used  in  the  follow-up  phase  of  data  collection.  Results  of  the  analysis  will  be  addressed 
in  Chapter  IV.  The  initial  analysis  was  conducted  following  the  CTA  framework.  The 
data  was  consolidated  into  narrative  that  could  be  further  grouped  into  aspects  of  the 
cognitive  tasks  that  could,  in  turn,  be  targeted  for  improvement.  These  steps  are  similar 
to  the  cognitive  inquiry  review  process,  which  enabled  the  blending  of  the  two 
approaches. 
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2. 


Consolidation 


Consolidation  consisted  of  transcription  of  interviews  and  spreadsheet  input  for 
qualitative  data.  Transcription  was  a  shared  effort  between  the  researcher  and  a 
professional  service.  Of  the  1 1  interviews,  the  researcher  transcribed  six,  three  were 
professionally  transcribed,  one  was  not  recorded  due  to  security,  and  the  other  was  not 
recorded  due  to  lack  of  consent.  The  researcher’s  method  of  transcription  was  to  listen  to 
the  cassette  tape,  or  digital  media  file  multiple  times  and  record  what  was  stated  in  a 
Microsoft  Word  document.  The  professional  transcription  service  returned  the  transcripts 
as  Microsoft  Word  documents  as  well.  For  the  interviews  that  were  not  recorded,  the 
researcher  typed  his  notes  into  another  Microsoft  Word  document.  When  the 
professional  transcripts  were  received,  the  researcher  consolidated  all  of  the  documents 
into  a  single  Microsoft  Word  document  for  follow  on  grouping. 

Surveys  consisted  of  responses  that  yielded  data  that  could  be  analyzed  both 
qualitatively  and  quantitatively.  Qualitative  data  was  consolidated  using  the 
consolidation  method  for  unrecorded  interviews  described  above.  Quantitative  data  was 
entered  into  a  spreadsheet  for  statistical  analysis. 

3.  Grouping 

Qualitative  data  was  grouped  into  tables  and  quantitative  data  underwent 

statistical  analysis.  The  qualitative  data  tables  extracted  and  consolidated  the  most 

significant  data  from  the  narrative  responses  from  interviews  and  surveys.  The  tables 

were  used  in  the  creation  of  follow-up  survey  questions  and  created  a  pool  of  notes  for 

use  in  the  creation  of  an  affinity  diagram  in  the  consolidated  analysis  phase.  The  process 

of  creating  the  qualitative  data  tables  consisted  of  integrating  the  answers  to  each 

question,  grouping  similar  responses  and  identifying  trends  in  the  groups.  Integrating  the 

answers  involved  assembling  the  answers  to  a  question  under  the  text  of  the  question 

itself.  Then,  all  of  the  responses  were  reviewed  together,  and  similar  responses  were 

grouped  under  a  single  phrase.  Here  is  where  the  lack  of  standardization  in  questioning 

affected  the  analysis.  Because  many  of  the  same  questions  were  omitted  in  multiple 

interviews,  those  lacking  answers  sometimes  outnumbered  those  with  answers,  which 
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reduced  their  significance.  Once  the  answers  were  grouped,  the  groups  were  reviewed 
for  trends  and  put  into  tables,  which  were  used  in  the  follow-up  and  consolidated 
analysis. 

Quantitative  data  was  run  through  the  descriptive  statistics  application  of  the  data 
analysis  add-on.  The  program  returned  1 5  statistics  for  each  question.  The  most  useful 
statistics  for  this  analysis  were  the  mean,  mode,  and  standard  deviation.  The  statistics 
will  be  discussed  in  the  following  chapter. 

E.  FOLLOW-UP  METHODOLOGY 

1.  Method  Determination 

A  second  round  of  interviews  to  target  information  gaps  found  in  the  initial 
analysis  was  expected  as  a  part  of  both  the  cognitive  task  analysis  and  contextual  inquiry 
methods.  The  decision  to  make  at  this  phase  was  whether  to  conduct  follow-up 
interviews  in  person  or  over  the  telephone.  Telephone  interviews  were  chosen  due  to  the 
researcher’s  experience,  nature  of  the  questions,  funding,  and  time.  Because  the 
researcher  experienced  multiple  work  environments  during  the  initial  interviews, 
revisiting  them  was  not  deemed  necessary.  The  nature  of  the  questions,  based  on  gaps 
found  in  the  initial  analysis,  did  not  require  physical  collocation  with  the  interviewee. 
Funding  may  have  been  made  available,  if  travel  was  necessary.  In  this  case,  however,  it 
was  decided  that  travel  was  not  a  high  enough  priority  to  allocate  funds.  The  amount  of 
time  required  for  both  coordination  and  travel  would  have  taken  the  researcher  from 
required  classes  and  slowed  the  production  of  this  document.  Telephone  interviews  were 
the  more  efficient  in  all  aspects. 

F.  FOLLOW-UP  PREPARATION 

1.  Participant  Selection 

Participants  selected  for  follow-up  interviews  were  either  personnel  within  the 
JTB  or  contractors  who  had  supported  the  JTB.  Factors  that  influenced  participant 
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selection  were  the  number  of  relevant  positions  in  the  organizations  and  the  participant’s 
perspective  on  information  sharing  between  the  organizations.  There  are  fewer  positions 
relevant  to  this  thesis  at  the  ranges  than  the  JTB,  due  to  the  nature  of  the  work.  At  the 
ranges,  the  majority  of  the  positions  focus  on  the  execution  of  tests  at  that  location,  so  the 
number  of  people  to  interview  who  have  experience  with  sharing  infonnation  between 
organizations  is  limited.  The  JTB  is  the  opposite.  The  majority  of  the  positions  deal  with 
sharing  information  with  other  organizations,  both  inside  and  outside  of  the  JTB.  The 
initial  round  of  interviews  adequately  covered  the  relevant  positions  at  organizations 
outside  the  JTB,  so  the  follow-up  interviews  were  focused  on  the  JTB  or  contractors  with 
experience  at  the  JTB  level.  The  perspective  of  information  sharing  at  the  JTB  level 
provided  more  information  on  sharing  between  organizations  than  individuals. 
Collecting  additional  data  from  that  perspective  was  appropriate  because  it  provided 
clearer  focus  on  the  scope  of  this  thesis. 

G.  FOLLOW-UP  DATA  COLLECTION 

1.  Follow-Up  Interviews 

Follow-up  interviews  consisted  of  17  questions  divided  into  five  topic  areas  that 
addressed  organizational  culture,  organizational  environment,  and  participatory  design. 
See  the  tables  in  the  results  chapter  for  the  text  of  the  questions. 

Topic  1,  specific  information  that  needs  to  be  shared,  consisted  of  four  questions. 
Questions  1.2  and  1.3  addressed  motivation,  and  1.1  and  1.4  addressed  participatory 
design.  The  flow  of  the  questions  within  Topic  1  identifies  a  specific  type  of  information 
that  needs  to  be  shared  and  extracts  the  individual’s  motivation  to  share.  Asking  the 
interviewee  to  speculate  about  why  others  do  not  share  that  infonnation  in  1.3  reveals 
more  about  his  motivation  to  share.  A  solicitation  for  recommendations  concludes  the 
topic  area. 

Topic  2,  information  visibility  between  organizations,  consisted  of  three 
questions.  Questions  2.1  and  2.2  addressed  the  personal  interaction  aspect  of  the 
organizational  environment,  and  2.3  addressed  the  norms  of  the  organizational  culture. 
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The  personal  interaction  questions  targeted  the  decision  making  process  of  finding  points 
of  contact  in  order  to  gain  visibility  regarding  a  piece  of  information  in  a  JTB 
suborganization.  Whether  or  not  attempting  to  gain  visibility  of  information  outside  of 
the  JTB  was  a  nonn  within  the  organization  was  relevant  to  assess  assertiveness  within 
the  organizational  culture. 

Topic  3,  organizational  structure,  consisted  of  four  questions.  Question  3.0 
addressed  participatory  design  and  the  others  addressed  the  organizational  environment. 
Identifying  the  customers  from  the  perspective  of  the  participants  provides  data  to  system 
engineers  on  which  organizations  or  billets  to  use  as  a  target  audience.  The  remaining 
questions  focused  on  trust  within  the  organizational  environment  in  order  to  detennine 
whether  or  not  it  should  be  targeted  as  an  area  for  improvement.  Trust  has  added 
significance,  due  to  the  JTB  being  a  nonhierarchical  organization. 

Topic  4,  specific  information  to  be  put  into  a  central  repository,  consisted  of  five 
questions.  Questions  4.1  and  4.2  addressed  participatory  design  and  4.3  through  4.5 
addressed  the  organizational  environment.  The  participatory  design  questions  solicited 
specific  data  points  on  what  type  of  information  and  content  should  be  included  in  a 
central  repository.  The  remaining  questions  dealt  with  trusting  information  from  that 
repository.  These  questions  differed  from  the  previous  topic  area  by  focusing  on 
information  rather  than  individuals. 

Topic  5,  security  classification  effects  on  information  sharing,  consisted  of  two 
questions.  Both  addressed  participatory  design.  Knowing  that  changing  security 
classification  was  beyond  the  authority  of  the  JTB,  the  questions  solicited  the 
interviewees’  view  on  what  barriers  were  created  by  security  classification  and 
recommendations  to  mitigate  them. 

a.  Follow-Up  Interview  Summary 

The  follow-up  round  consisted  of  four  telephone  interviews.  The  general 
flow  of  the  interviews  included  introductions,  research  overview,  consent  form 
confirmation,  survey  receipt  verification,  and  interview  questions. 


26 


2. 


Follow-Up  Surveys 


The  follow-up  surveys  were  unchanged  from  the  initial  round  of  interviews  and 
were  administered  to  all  participants.  The  same  content  in  the  surveys  was  used  for 
consistency  and  because  the  experienced  participants  would  provide  quality  input.  The 
only  change  in  the  survey  was  the  format.  Both  surveys  were  altered  to  have  the 
interviewees  fill  in  fonn  fields  rather  than  fill  them  in  manually.  The  alterations  made  it 
easier  to  return  to  the  researcher  via  e-mail.  Consent  forms  and  both  surveys  were  e- 
mailed  to  all  four  applicants.  All  consent  fonns  were  signed  and  returned,  but  some 
surveys  were  not  returned.  Details  are  included  in  the  tables  in  the  results  chapter. 

The  difference  in  the  return  rate  for  the  initial  in  person  and  follow-up  e-mailed 
survey  responses  was  significant.  Follow-on  researchers  are  advised  to  make  every  effort 
to  administer  surveys  in  person  in  order  to  insure  a  response. 

H.  FOLLOW-UP  ANALYSIS 

1.  Consolidation  and  Grouping 

The  data  for  the  follow-up  interviews  was  consolidated  and  grouped  using  the 
same  methodology  as  the  initial  interviews.  All  interview  transcriptions  were  done  by  the 
researcher.  The  interview  results  were  grouped  independently  of  the  initial  interviews 
because  the  questions  were  different.  The  initial  and  follow-up  survey  results  were 
consolidated  because  the  questions  were  the  same  for  both  rounds  of  data  collection. 

I.  CONSOLIDATED  ANALYSIS 

1.  Affinity  Diagram 

The  affinity  diagram  was  used  to  further  refine  the  results  of  the  initial  and 
follow-up  phases  of  interviews  into  a  fonnat  specifically  designed  for  cognitive  systems 
engineering.  The  complete  affinity  diagram  is  presented  in  the  results  section. 
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a.  Establish  a  Focus 

The  first  step  in  creating  the  affinity  diagram  was  to  establish  a  focus.  The 
primary  focus  for  this  affinity  diagram  was  improving  information  sharing  within  the 
JTB,  as  defined  for  this  thesis.  The  researcher  posed  the  following  questions  to  himself 
throughout  the  creation  process:  “What  barriers  to  infonnation  sharing  exist?”;  “How  can 
the  barriers  be  mitigated?”;  “What  recommendations  for  improvement  were  provided  by 
the  users?” 


b.  Record  Key  Aspects 

With  the  focus  questions  in  mind,  the  researcher  recorded  key  aspects 
from  the  initial  and  follow-up  analysis  phases  on  a  Microsoft  Excel  spreadsheet.  The 
cognitive  task  analysis  tables  created  from  the  interviews  and  qualitative  survey  questions 
in  the  earlier  analysis  phases  provided  the  key  aspects  included  in  the  diagram. 

Each  key  aspect  was  recorded  in  a  cell  on  the  spreadsheet  labeled  Tier  III. 
The  headers  were  reorganized  and  categorized  on  a  spreadsheet  labeled  Tier  II.  The  Tier 
II  headers  were  reorganized  and  categorized  on  a  spreadsheet  labeled  Tier  I.  The 
organization  of  an  example  affinity  diagram  is  presented  in  Figure  3. 
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Tier  I 


Tier  II 


Tier  III 


Figure  3.  Example  of  an  Affinity  Diagram 


Although  adhesive  notes  are  the  recommended  material  to  create  an 
affinity  diagram,  the  electronic  format  was  more  applicable  to  this  situation  because  it 
was  being  recorded  by  an  individual  and  the  results  would  have  to  be  shared.  As  an 
individual,  using  the  electronic  format  was  less  cumbersome  and  more  mobile  than  the 
adhesive  note  method.  Also,  because  the  final  product  will  be  provided  to  the  follow  on 
researcher  conducting  research  on  JTB  portal  design,  an  electronic  fonnat  was  easier  for 
this  researcher  to  reproduce  and  the  follow-on  researcher  to  manipulate. 

c.  Tier  III 

After  all  aspects  were  entered  into  cells,  they  were  grouped  into  thirteen 
columns,  and  each  group  was  given  a  heading.  The  headings  represented  the  main  points 
in  each  column,  as  related  to  the  focus.  The  headings  on  Tier  III  were:  Personal 
Relationships,  Individual  Perceptions,  JTB  Level  Issues,  Leadership,  Inter-Organizational 
Communication,  Best  Practices,  Standards,  Points  of  Contact,  Personnel  Turnover, 
Quality  of  Shared  Infonnation,  Information  Sharing  Tools,  Access  to  Information,  and 
Information  Visibility. 

d.  Tier  II 

The  Tier  III  headings  became  the  key  aspects  for  two  further  groupings. 
They  were  copied  and  pasted  onto  another  spreadsheet,  labeled  Tier  II.  This  was  done  in 

29 


order  to  enhance  focus  solely  on  the  group  headings,  make  the  diagram  less  cumbersome, 
and  make  the  final  product  more  organized  and  readable.  The  headings  were  grouped 
and  given  their  own  heading  in  the  same  manner  as  the  Tier  III  aspects.  The  headings  on 
Tier  II  were:  Individual  Issues,  Leadership  Considerations,  Management,  Information 
Sharing,  and  Information  Availability. 

e.  Tier  I 

The  Tier  II  headings  were  copied  and  pasted  onto  a  spreadsheet  labeled 
Tier  I,  in  the  same  manner  as  previously  described.  The  Tier  II  headings  were  then 
grouped  and  given  another  heading.  The  Tier  I  headings  were  the  top-level  headings  for 
the  affinity  diagram.  The  Tier  I  headings  were:  Individuals,  Leadership,  and  Infonnation. 

f  Record  the  Diagram 

Recording  the  diagram  was  not  an  issue,  due  to  the  use  of  the  electronic 

format. 


2.  Surveys 

a.  Qualitative  Data 

The  qualitative  data  from  the  initial  and  follow-up  interview  question 
analysis  was  included  in  Tier  III  of  the  affinity  diagram. 

b.  Quantitative  Data 

The  quantitative  data  from  the  initial  and  follow-up  infonnation  sharing 
survey  analysis  was  consolidated  and  run  through  the  descriptive  statistics  application  of 
the  data  analysis  add-on.  The  program  returned  1 5  statistics  for  each  question.  The  most 
useful  for  this  analysis  were  the  mean,  mode,  and  standard  deviation.  The  consolidated 
statistics  will  be  discussed  in  the  following  chapter. 
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J.  SUMMARY 

1.  Overview 

The  research  methods  for  cognitive  task  analysis  and  contextual  inquiry  were 
incorporated  for  this  thesis.  Data  collection  consisted  of  structured  interviews,  a 
demographic  survey,  and  an  infonnation  sharing  survey.  Questions  were  focused  on 
individual  motivation,  organizational  influences,  and  cognitive  engineering.  Data 
analysis  included  the  creation  of  data  tables,  in  accordance  with  cognitive  task  analysis, 
an  affinity  diagram,  in  accordance  with  contextual  inquiry,  and  statistical  analysis. 

2.  Data  Collection 

a.  Interviews 

Data  collection  consisted  of  structured  interviews  and  surveys.  The 
interviews  were  conducted  in  two  rounds,  initial  and  follow-up,  each  with  different 
questions.  The  initial  questions  were  conducted  in  person  at  YPG,  China  Lake,  APG, 
JTB  offices,  Point  Mugu,  and  NPS.  Follow-up  interviews  were  conducted  via  telephone 
with  JTB  members  and  contractors  with  JTB  level  experience. 

b.  Surveys 

Two  surveys  were  administered,  a  demographic  survey,  and  an 
information  sharing  survey.  Both  were  administered  in  two  rounds,  initial  and  follow-up, 
with  the  same  content,  to  the  same  people  that  were  interviewed.  The  initial  round  was 
administered  in  person  and  all  responses  were  gathered.  The  follow-up  round  was 
administered  via  e-mail  and  multiple  responses  were  not  returned. 
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3. 


Data  Analysis 


a.  Qualitative 

The  qualitative  information  from  each  round  of  interviews  and  the 
qualitative  questions  from  the  surveys  were  used  to  create  tables  that  grouped  the  data 
into  cohesive  data  points  that  revealed  the  cognitive  nature  of  the  tasks  involved. 
Separate  tables  were  created  for  each  round.  Those  data  points  were  then  consolidated 
and  used  as  the  key  aspects  in  the  creation  of  a  three-tier  affinity  diagram,  which  will  be 
provided  to  a  follow  on  researcher  conducting  research  on  JTB  portal  design. 

b.  Quantitative 

Quantitative  infonnation  from  surveys  from  each  round  was  run  through 
statistical  analysis  software  separately.  The  data  was  then  consolidated  and  run  through 
the  same  software.  The  initial,  follow-up,  and  consolidated  statistics  will  be  discussed  in 
the  following  chapter. 
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IV.  RESULTS 


A.  OVERVIEW  OF  RESULTS 

1.  Presentation  of  Results 

This  chapter  presents  the  results  of  the  research.  The  methodologies  used  were 
designed  to  produce  tabular  results,  which  create  the  majority  of  the  chapter.  Text  is  used 
minimally  to  provide  context  for  tabular  content  or  present  results  that  are  not  suitable  for 
tabular  presentation. 

2.  Consolidated  Demographics 

The  demographic  information  for  the  15  interviewees  shows  that  they  are  well 
educated  and  experienced,  especially  considering  the  relatively  short  existence  of  the 
JTB.  It  also  shows  the  diversity  in  duties  among  the  members  of  the  organization,  which 
is  reflected  in  the  interview  results  as  well.  Due  to  the  number  of  demographic  responses 
that  were  not  appropriate  for  tabular  presentation,  the  results  will  be  described  in  the 
following  paragraphs. 

The  mean  age  group  of  the  participants  was  31  to  40  years  old.  Fourteen  of  the 
15  participants  were  male.  The  education  level  of  the  participants  ranged  from  high 
school  to  doctorate  degree,  with  the  majority  of  the  participants  having  a  bachelor’s 
degree.  Seven  of  15  had  taken  a  single  fonnal  course  relative  to  C-IED  testing,  and  one 
had  taken  several  related  courses. 

Twelve  of  15  participants  had  testing  experience.  The  three  participants  without 
testing  experience  were  JTB  employees  not  directly  involved  in  testing.  Of  those  with 
experience,  the  average  length  of  experience  with  C-IED  testing  was  4.8  years.  Five 
participants  were  in  positions  directly  involved  with  testing,  and  10  were  not.  The 
average  time  in  their  current  position  was  one  to  five  years,  one  had  less  than  one  year 
and  two  had  three-  to  ten-year’s  experience. 
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Regarding  their  place  of  employment,  six  were  from  the  JTB,  three  were  involved 
in  testing  at  YPG,  one  was  from  the  Advanced  Communications  working  group,  one  was 
from  China  Lake,  one  was  from  a  program  office,  and  three  were  not  associated  with  the 
JTB. 

Five  of  the  15  had  worked  as  members  of  multiple  JTB  organizations.  One  at  the 
JTB  had  served  on  the  National  Assessment  Group  and  as  a  technical  advisor,  one  from  a 
test  range  listed  another  position  at  the  same  range,  one  in  the  Advanced 
Communications  Working  Group  had  experience  at  a  test  range,  one  at  a  program  office 
listed  another  C-IED  related  program  office,  and  one  provided  no  further  details.  Of  the 
15,  seven  had  not  participated  in  any  JTB  sponsored  tests,  two  participated  in  less  than 
10,  two  participated  in  between  90  and  100,  and  four  participated  in  200  or  more  tests. 

3.  Interview  Summary 

a.  Initial  Interview  Summary 

The  initial  interview  questions  were  separated  in  two  sections,  reflected  in 
the  two  tables  that  follow.  The  tables  include  the  question  number,  question  text,  a 
summarized  analysis  of  the  answers,  and  the  number  of  responses  for  each  question. 
Detailed  data  points  from  the  interviews  will  be  presented  in  the  CTA  tables  and  the 
affinity  diagram  later  in  this  chapter. 

Table  2  presents  responses  to  the  survey  questions  in  the  Information 
Sharing  Section,  which  focused  on  the  individuals’  experiences  with  sharing  information. 


Table  2.  Initial  Interview  Infonnation  Sharing  Section  Summary 


# 

Question  Text 

Answer  Summary 

N* 

1 

From  your  perspective,  what 
differences  exist  regarding 
information  needs  between  the 
organizations? 

Information  needs  are  similar  among  the  test  facilities. 
However,  organizations  that  process  and  distribute  the 
results  have  different  requirements. 

11 

2 

Please  describe  a  time  when  you 
needed  to  use  information  from 
another  organization. 

All  examples  ended  in  receiving  the  required  information. 
Only  one  dealt  with  another  organization  within  the  JTB, 
and  that  was  contingent  on  a  personal  relationship. 

5 
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3 

Please  describe  a  time  when  you 
were  asked  to  share  your 
information  with  another 
organization. 

Examples  included  inter-testing  facility  sharing 

5 

4 

Please  describe  an  exceptionally 
challenging  experience  when  you 
successfully  or  unsuccessfully 
shared  information  with  the  JTB  or 
one  of  its  organizations. 

Positive  feedback  focused  on  cooperation  between  test 
ranges.  Negative  feedback  referred  to  information 
visibility  and  availability,  changing  requirements,  getting 
raw  data  from  a  test  facility,  and  sharing  with  coalition 
partners. 

10 

5 

In  your  experience  with  JTB  tests, 
to  what  extent  does  your 
organization  provide  information  to 
other  organizations? 

Information  is  provided  to  others  through  personal  points 
of  contact  and  by  posting  it  to  the  JTB  portal  and  Theater 
Support  Web  Tool  (TSWT).  Answers  did  not  explicitly 
address  the  extent  of  providing  information. 

10 

6 

When  was  the  last  time  you  or  your 
team  provided  information  to 
another  organization?  How  did  you 
send  that  information?  Is  that 
typical?  What  is  typical? 

Information  provision  ranged  from  constantly  to  semi¬ 
annually,  with  a  majority  being  daily.  All  means  were 
typical  and  included  personal  communication,  e-mail, 

JTB  Portal,  video  teleconference,  and  telephone. 

11 

7 

In  your  experience  with  JTB  tests, 
to  what  extent  does  your 
organization  receive  information 
from  other  organizations? 

Information  reception  ranged  from  none  to  ‘as  it  is 
requested’,  with  a  majority  being  ‘as  it  is  requested’.  A 
single  response  included  receiving  information  without 
requesting  it,  and  that  dealt  with  budgeting. 

11 

8 

When  was  the  last  time  you  or  your 
team  used  information  received 
from  another  organization?  How 
did  you  receive  that  information?  Is 
that  typical?  What  is  typical? 

Timeframes  ranged  from  currently  to  a  couple  of  months 
ago.  Noted  reception  methods  included  e-mail,  video 
teleconference,  and  the  JTB  database. 

11 

9 

In  your  experience,  how  much  of 
the  information  you  needed  from 
other  organizations  have  you  been 
able  to  obtain?  To  the  extent  that 
you  were  unable  to  obtain  all  of  the 
information  you  needed,  what  was 
the  primary  cause  of  the  problem? 

Information  obtain  ability  ranged  between  ‘a  fair  amount’ 
to  all  of  it,  with  a  single  exception  the  replied  ‘no’. 
Problems  consisted  of  a  lack  of  timeliness,  lack  of  access 
to  the  information,  managerial  resistance,  and  reception 
being  based  on  inconsistent  personal  relationships 
because  the  organization  is  always  changing. 

11 

10 

What  are  the  most  significant 
aspects,  positive  or  negative,  of 
sharing  information  between  JTB 
organizations? 

Positive:  ranges  working  together;  sharing  once  makes  it 
easier  the  second  time;  and  everything  is  done  for  the  war 
fighter.  Negative:  people  don’t  know  who  to  ask  for  what 
type  of  information;  people  at  different  locations  do  not 
know  each  other;  and  the  receiver  must  understand  the 
information  before  it  should  be  shared. 

6 

11 

Does  your  organization  share 
information  with  other 
organizations  through  ad  hoc 
means  or  means  other  than  those 
provided  through  the  JTB?  If  yes, 
what  are  they?  How  well  do  you 
think  they  would  work  if  they  were 
used  throughout  the  JTB  and  other 
organizations? 

No  ad  hoc  methods  were  provided.  Means  provided 
included  working  groups,  video  teleconferences,  the  JTB 
Portal,  and  test  protocols. 

6 
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12 

Exchanging  more  information  with 
which  other  JTB  organization 
would  improve  your  organization’s 
performance  (examples  if  needed: 
test  range,  working  group,  board 
members)? 

The  majority  of  responses  did  not  answer  the  question, 
but  were  addressed  in  the  analysis.  Relevant  responses 
were  other  test  ranges,  JIEDDO,  and  JEIDDO  modeling 
and  simulation. 

11 

13 

What  changes  would  you  make  to 
improve  information  sharing 
between  all  personnel  involved  in 
JTB  testing? 

Recommendations  related  to  the  following:  a  better 
database,  ensuring  awareness  of  available  information, 
ensure  the  principle  players  are  coordinating  with  each 
other,  disseminate  working  group  information,  and 
disseminate  an  up-to-date  organizational  chart. 

11 

14 

What,  if  any,  impediments  to 
sharing  test  results  are  you  aware 
of? 

Impediments  consisted  of:  corporate  culture,  financial 
competitiveness  of  the  ranges,  inability  to  access  other 
range  databases,  approval  required  before  information  can 
be  released  (one  location),  results  from  other  customer 
tests  are  not  shared,  and  foreign  release  ability 

10 

15 

Please  describe  any  information 
gaps  that  exist  regarding 
information  sharing  and  how  you 
would  fill  them 

Four  of  the  eight  responses  noted  the  absence  of  gaps. 
Others  made  the  following  recommendations:  create  an 
information  czar  to  manage  working  group  information; 
make  PMS-408  the  Joint  Counter  Radio-Controlled 
Electronic  Warfare  Program  Manager;  and  standardize 
data  collection  and  storage  (file  formats,  etc.). 

8 

16 

Would  more  sharing  of  information 
between  organizations  help  or 
hinder  timely  and  accurate  results? 
How? 

All  responded  that  it  would  help,  but  none  described  how. 

2 

*Number  of  responses  out  of  1 1 


Table  3  presents  the  results  of  the  participant  responses  to  items  in  the 
Policy  and  Procedure  Section,  which  focused  on  sharing  between  organizations  and  the 
guidance  from  the  top  of  the  hierarchy. 


Table  3.  Initial  Interview  Policy  and  Procedure  Section  Summary 


# 

Question  Text 

Answer  Summary 

N* 

1 

How  does  the  information  that  you 
process  or  create  get  to  the  JTB? 

Responses  consisted  of  the  following:  it  depends  on  the 
personal  relationships  involved,  directly  to  the  JTB,  post 
to  the  JTB  portal,  directly  to  foreign  release  officer,  post 
to  Theater  Support  Web  Tool  (TSWT),  send  it  through  a 
working  group 

6 

2 

What  specifically  enables  or  inhibits 
information  sharing  with  the  JTB 
and  other  organizations? 

Culture,  personal  bias,  nothing,  lack  of  visibility  on 
what’s  been  collected,  security  classification 

5 

3 

What  procedural  improvements  do 
you  recommend  to  mitigate  any 
inhibitions  to  information  sharing? 

Recommendations  consisted  of  the  following:  bring  all  C- 
IED  testing  under  the  JTB’s  purview,  current  process  is 
good,  (oracle)  database  will  help,  all  data  from  all 
locations  should  flow  to  a  backup  site,  create  standard 
operating  procedure  for  information  sharing,  and  I  don’t 

10 
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know 

4 

Is  sharing  information  between 
organizations  and  the  JTB  supported 
by  clear  policy  and  guidance?  In 
your  experience,  is  this  policy 
routinely  enforced? 

Responses  included:  guidance  is  there,  guidance  is  not 
there,  there  is  a  policy,  there  is  probably  a  policy  -  but  I 
don’t  know  what  it  is,  there  is  not  policy,  and  people 
follow  norms  rather  than  written  policy 

11 

5 

What  policy  changes,  if  any,  would 
you  recommend  to  enhance 
information  sharing  between 
organizations  and  the  JTB? 

Recommendations  were:  have  ranges  work  together  on 
tests,  create  a  database  with  the  ranges  and  their 
customers,  all  JTB  members  should  have  to  visit  the  test 
ranges  to  establish  personal  contacts,  create  turnover 
binders,  and  enforce  existing  policy 

10 

6 

Please  explain  any  uncertainty  or 
confusion  regarding  information 
flow  into  or  out  of  your 
organization.  In  your  opinion,  how 
can  the  uncertainty  or  confusion  be 
mitigated? 

Of  those  who  responded  that  there  is  uncertainty,  it  is  due 
to:  test  ranges  have  other  customers  besides  the  JTB, 
personnel  turnover  at  JTB  requires  the  establishment  of 
new  personal  relationships,  and  people  don’t  know  who  to 
go  to  for  what  information. 

Recommendations  for  mitigation  included:  process  maps 
for  data  flow  processes,  infrastructure,  research  & 
development,  get  employees  together  to  build  personal 
relationships,  and  creating  an  information  sharing  policy 

10 

7 

Please  describe  what  you  think 
would  be  the  most  efficient  route 
and  method  to  exchange 
information  between  organizations. 

Recommendations  included:  add  a  pop-up  alert  for  new 
information  on  the  JTB  portal  and  implement  a  quality 
easy  to  use  web  interface 

3 

*Number  of  responses  out  of  1 1 


b.  Follow-Up  Interview  Summary 

The  follow-up  interview  format  was  separated  into  five  topics,  reflected  in 
the  following  live  tables.  Topic  one  questions  addressed  specific  information  that  needs 
to  be  shared.  The  questions  under  this  topic  were  meant  to  clarify  preliminary  analysis  of 
the  initial  survey  results,  which  showed  that  interviewees  were  receiving  all  of  the 
information  that  they  needed,  but  thought  more  sharing  would  be  beneficial.  While  many 
good  data  points  were  raised  and  included  in  the  CTA  and  affinity  diagram,  not  all  of  the 
specific  details  that  were  targeted  by  the  questions  were  provided. 


Table  4.  Follow-Up  Interview  Summary:  Specific  Information  That  Needs  to  be 

Shared 


Topic  1  -  Specific  Information  that  Needs  to  be  Shared 

# 

Question  Text 

Answer  Summary 

N* 

1 

What  specific  type  of  information 
needs  to  be  shared? 

Most  of  the  responses  did  not  answer  the  question,  but 
were  included  in  the  analysis.  The  one  response  that  did 

4 
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answer  the  question  recommended  that  data  being 
collected  by  the  program  offices  should  be  shared  with 
the  JTB  and  the  field. 

2 

Considering  your  answer  to  the 
previous  question,  why  are  you  not 
giving  that  specific  information? 

Responses  were  either  that  the  question  did  not  apply 
because  the  interviewees  were  not  sharing  the  information 
or  did  not  directly  answer  the  questions,  but  were 
included  in  the  overall  analysis. 

4 

3 

Regarding  that  same  information, 
why  do  you  think  that  you  are  not 
receiving  it? 

Suggestions  included:  a  lack  of  trust,  the  difficulties 
involved,  the  lack  of  a  process,  and  some  people  don’t 
perceive  a  requirement  to  share 

4 

4 

How  can  the  situation  be  remedied 
to  ensure  people  are  able  to  share 
the  right  information? 

Recommendations  included:  post  information  to  a  single 
location  to  ease  access,  ensure  visibility  of  information, 
increase  coordination,  create  a  better  process  for  sharing, 
and  conduct  working  groups  to  integrate  the  methods  of 
all  locations. 

4 

5 

Who  are  the  JTB’s  customers? 

DoD,  operating  forces,  coalition  partners 

4 

*Number  of  responses  out  of  four 


Topic  two  questions  addressed  information  visibility  between 
organizations.  Issues  raised  during  initial  interviews  included  of  a  reliance  on  personal 
relationships  for  sharing  information,  problems  being  able  to  figure  out  who  to  contact  to 
get  information,  and  issues  with  visibility  of  who  has  what  information.  The  visibility 
issue  referred  to  both  within  the  JTB  and  outside,  including  program  managers  and  other 
range  customers.  Therefore,  questions  related  to  this  topic  focused  on  how  interviewees 
used  existing  points  of  contact  and  how  they  created  new  contacts  in  order  to  discover 
information  that  was  not  readily  visible  and  accessible. 


Table  5.  Follow-Up  Interview  Summary:  Infonnation  Visibility  Between 

Organizations 


Topic  2  -  Information  Visibility  Between  Organizations 

# 

Question  Text 

Answer  Summary 

N* 

1 

When  you  need  a  piece  of 
information,  please  describe  the 
process  you  use  to  decide  who  to 
ask  for  it? 

Processes  included:  use  the  organizational  chart,  use  the 
JTB  portal  to  find  the  information  without  asking 
someone,  and  weekly  video  teleconferences 

4 

2 

If  you  have  a  need  for  information, 
but  do  not  have  a  personal 
relationship  with  someone  who  has 
it,  how  do  you  find  out  who  to 
contact  to  get  the  information  you 
need? 

The  answers  were  the  same  as  question  1 . 

4 

3 

Understanding  that  organizations 
outside  of  the  JTB  can  have 

Three  of  the  four  answered  yes,  while  the  other  answered 
that  it  should  be  taken  care  of  through  the  contracting 

4 
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proprietary  information,  if  you  had 

process.  Means  of  finding  points  of  contact  consisted  of 

reason  to  believe  that  someone 

networking  through  working  groups  and  at  conferences, 

outside  of  the  JTB,  such  as  a 

getting  them  from  the  chain  of  command,  and  calling  a 

program  manager  or  civilian 

business  contact  number  and  talking  to  people  within  the 

business,  had  information  that  could 
benefit  your  organization,  would 
you  contact  them? 

organization. 

*Number  of  responses  out  of  four 


Topic  three  questions  addressed  organizational  structure.  The  JTB  does 
not  fit  the  mold  of  a  hierarchical  organization,  due  to  the  geographically  separated  nature 
of  the  enterprise,  which  includes  a  collection  of  different  services  and  agencies 
conducting  different,  but  similar  tasks.  As  a  result,  the  JTB  is  partially  a  flat 
organization.  In  order  for  flat  organizations  to  work,  the  employees  must  trust  each  other 
and/or  the  information  the  other  provides.  The  questions  under  this  topic  focused  on 
trust. 


Table  6.  Follow-Up  Interview  Summary:  Organizational  Structure 


Topic  3  -  Organizational  Structure 

# 

Question  Text 

Answer  Summary 

N* 

1 

Do  you  trust  the  information  that 
you  receive,  whether  or  not  you 
have  met  the  person  sending  it? 

The  majority  said  yes,  but  the  others  would  check  the 
sender’s  reputation  with  others  in  the  same  field. 

4 

2 

Do  you  trust  other  members  of  the 
JTB  organization  enough  to  share 
information  with  them,  even  if  you 
have  never  met  them  face  to  face? 

The  overall  consensus  was  yes,  if  they  had  a  need  to 
know. 

4 

3 

Do  you  consider  other  members  of 
the  JTB  organization  experts  in  their 
particular  fields? 

Half  would  and  the  other  half  would  want  to  confirm  their 
credibility  before  assuming  they  were  experts. 

4 

*Number  of  responses  out  of  four 


Topic  four  questions  addressed  specific  information  to  be  put  into  a 
central  repository.  The  purpose  of  this  topic  was  to  elicit  specifics  regarding  the  content 
and  use  of  a  central  repository  because  the  creation  of  ‘a  database’  was  a  recurring 
recommendation  by  multiple  interviewees  for  many  topics  in  the  initial  round  of 
interviews. 
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Table  7.  Follow-Up  Interview  Summary:  Specific  Information  to  be  Put  Into  a 

Central  Repository 


Topic  4  -  Specific  Information  to  be  Put  into  a  Central  Repository 

# 

Question  Text 

Answer  Summary 

N* 

1 

What,  specifically,  should  be  in  a 
central  information  repository  for 
the  JTB? 

Data  that  is  appropriate  for  wide  distribution,  including: 
test  plans,  test  reports,  scenario  data, 
conditions/environmental,  performance  data,  and 
representation  of  the  sensor  for  instrumentation 

4 

2 

Should  the  repository  consist  of 
both  technical  and  administrative 
information  (scheduling,  budgeting, 
etc)? 

Responses  were  evenly  split  between  making  it  a  one-stop 
shop  for  all  information  and  making  it  a  strictly  technical 
store  of  knowledge 

4 

3 

Do  you  believe  that  others  would 
use  your  information  correctly? 

Why? 

All  agreed  that  the  information  would  be  used  correctly. 
Some  additional  information  included  the  need  for  a 
defined  organization  for  the  information  and  feedback 
mechanisms  on  the  JTB  portal. 

4 

4 

Do  you  trust  others  to  use  your 
information?  Why? 

Three  of  the  four  stated  yes,  and  the  other  noted  that  it 
had  to  be  made  available  and  that  most  people  would  use 
it  correctly 

4 

5 

Would  you  trust  the  information 
others  entered  into  the  central 
repository?  Why? 

All  agreed  that  they  would  trust  the  information.  Access 
controls  provided  additional  confidence  for  one  responder 

4 

*Number  of  Responses  out  of  four 


Topic  five  questions  addressed  the  effects  of  security  classification  on 
information  sharing.  Multiple  initial  interview  responses  stated  that  security 
classification  inhibited  information  sharing,  so  the  questions  under  this  topic  were  meant 
to  provide  clarification. 


Table  8.  Follow-Up  Interview  Summary:  Security  Classification  Effects  on 

Information  Sharing 


Topic  5  -  Security  Classification  Effects  on  Information  Sharing 

# 

Question  Text 

Answer  Summary 

N* 

1 

What  makes  the  security  classification  of 
a  report  an  inhibition  to  information 
sharing? 

Security  requirements  are  there  for  a  reason  and  must  be  followed 
regardless  of  the  situation.  If  a  report  is  to  be  shared  with  a 
coalition  partner,  at  least  two  different  copies  must  be  processed. 

4 

2 

What  are  your  suggestions  to  resolve  the 
issue? 

If  multiple  security  versions  will  be  required,  prepare  them 
simultaneously  from  the  start,  rather  than  editing  the  finished 
product 

4 

*Number  of  Responses  out  of  four 
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B.  QUALITATIVE  RESULTS 


1.  Cognitive  Task  Analysis 

a.  CTA  Table  Description 

This  section  describes  which  data  points  were  included  in  the  CTA,  how 
they  were  organized  and  presented,  and  acknowledges  the  absence  of  data  points  in 
specific  areas.  The  CTA  tables  include  data  points  from  initial  interviews,  follow-up 
interviews,  and  qualitative  answers  to  survey  questions. 

The  data  points  were  initially  organized  by  whether  they  identified  barriers 
or  enablers  to  information  sharing,  or  recommendations  to  improve  information  sharing. 
Those  data  point  groupings  were  then  subdivided  into  the  motivational  and  organizational 
culture,  environment,  and  structure  influences  described  in  the  methodology  chapter.  The 
result  was  a  collection  of  twelve  tables  that  organized  the  data  points  by  cause  (the 
methodology  chapter  organization),  and  effect  (barrier,  enabler,  or  recommendation). 

The  data  points  did  not  address  all  subdivisions,  so  some  of  the  columns 
have  no  entries,  and  are  annotated  accordingly.  Also,  three  tables  had  no  data  points,  so 
they  are  not  included;  they  were:  organizational  structure  enablers,  motivational 
recommendations,  and  organizational  structure  recommendations. 

Table  9  presents  the  motivational  barriers  based  on  information  collected 
from  participant  surveys.  It  includes  participant  responses  related  to  intrinsic,  intrinsic 
hedonic,  and  extrinsic  motivation. 


Table  9.  Motivational  Barriers 


Intrinsic 

Intrinsic  Hedonic 

Extrinsic 

Meetings  that  include  coalition 
partners  restrict  some 
communications 

Maintain  organizational 
reputation  by  preventing  misuse 
of  data  by  other  organizations 

Everyone  trying  to  secure 
funding 

People  are  reserved  in  large  video 
teleconferences  because  they  are 
not  willing  to  share  with 
information  with  all  of  the 
participants 

Maintain  organizational 
reputation  by  the  provider 
restricting  sharing  to  only  those 
who  he  believes  understand  what 
to  do  with  the  information 

Sharing  information  is  helping 
your  competition,  which  puts 
your  job  at  risk 
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Individuals  hoard  information  to 
make  themselves  seem  more 
powerful 

Testers  and  range  workers  fear 
losing  their  jobs 

Individuals  don't  share  in  order  to 
avoid  being  judged  by  their  peers. 

The  belief  that  the  info  is  mine  so 
no  one  else  can  interpret  it 

Individual's  fear  of  data  being 
misused 

Protect  the  war  fighter  by 
ensuring  that  information  is  not 
taken  out  of  context 

Table  10  presents  the  organizational  culture  barriers  based  on  information 
collected  from  participant  surveys.  It  includes  participant  responses  related  to  beliefs  and 
norms.  Belief  data  points  dealt  with  issues  related  to  policy  and  norm  data  points  dealt 
with  issues  related  to  process. 


Table  10.  Organizational  Culture  Barriers 


Beliefs  (Relating  to  Policy) 

Norms  (Relating  to  Process) 

Managerial  resistance  to  obtaining  data  from 
another  organization 

There  is  no  process  in  place  to  share  information 

Institutionalized  financial  resource  competition 
between  test  ranges  stops  information  sharing 

Most  tests  are  set  up  as  individual  tests  to  be 
conducted  at  a  single  range. 

Test  ranges  have  different  priorities  than  the  JTB 
because  they  have  to  work  with  other  customers  to 
obtain  additional  financial  resources 

Some  organizations  attempt  to  repress  certain 
information  if  it  might  be  harmful  to  a  system 

Rivalries  between  sites  and  services  create  barriers 

Lack  of  visibility  on  updated  documents 

Bureaucracy  has  made  the  organization  ineffective 

Inability  to  locate  documents  that  are  known  to  exist 
in  other  organizations 

Management  is  inadequate  at  multiple  levels 

Falsely  assuming  that  others  have  access  to  the 
same  information 

Decisions  are  made  in  a  risk  averse  environment 

Individuals  don’t  know  who  has  what  information 

Belief  throughout  the  organization  that  sharing 
information  may  result  in  losing  business  to  another 
organization 

Obtaining  raw  unfiltered  data  from  a  test  range  is  a 
challenge 

No  commonality  of  a  strategic  plan  between  ranges 
or  from  higher 

Requirements  continually  change 

There  is  no  standard  method  or  format  for 
requesting,  collecting,  storing,  or  disseminating  data 

Information  received  is  not  normally  timely  or 
accurate,  which  leads  to  getting  the  information 
through  different  means 

There  is  are  no  formal  procedures  for  information 
sharing 

When  sharing  occurs,  the  amount  of  information 
received  is  inadequate 

The  lack  of  a  written  Information  Sharing  policy 
results  in  confusion  through  reliance  on  a  set  of 
unwritten  norms  that  individuals  interpret 
differently 

People  get  the  information  they  need  only  after 
working  through  barriers 
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No  common  data  schema  or  data  collection  formats 

Requirements  levied  on  individuals  change 
frequently 

Sometimes  an  ultimatum  is  necessary,  even  if  others 
know  the  information  is  required 

The  corporate  culture  focuses  on  individuals  gaining 
power  over  others 

A  certain  test  range  will  share  only  complete  test 
reports  officially  approved  for  release 

The  corporate  culture  does  not  value  information 
sharing 

Much  of  the  information  gathered  is  proprietary  and 
not  shared 

Individuals  don’t  see  themselves  as  part  of  an 
enterprise 

There  is  a  lack  of  access  to  information  from  other 
facilities  and  ranges 

There  is  a  lack  of  access  to  documents  that  are 
known  to  be  available  to  others 

Table  1 1  presents  the  organizational  environment  barriers  based  on 
information  collected  from  participant  surveys.  These  include  participant  responses 
related  to  personal  interaction,  information  systems,  and  incentives. 


Table  1 1 .  Organizational  Environment  Barriers 


Personal  Interaction 

Information  Systems 

Incentive 

Sharing  is  often  contingent  upon 
who  you  know 

Theater  Support  Web  Tool  is  not 
user  friendly 

Nothing  Noted 

People  don’t  know  all  the 
individuals  and  personalities,  so 
they  have  a  difficult  time 
receiving  the  information  they 
need 

The  enterprise  lacks  a  good 
central  repository  that  has  all  the 
metadata  and  the  data  required  to 
make  use  of  the  information 

Some  individuals  share  only  what 
is  requested,  despite  knowing 
more  is  required 

There  are  issues  with  permissions 
in  the  information  system 

Working  group  members  don’t 
disseminate  what  they  learn  with 
co-workers 

Not  being  able  to  access  other 
ranges  database  for  information. 

Sharing  is  based  on  the  'good  ole 
boy'  system 

Individual  biases 

Lack  of  trust  throughout  the 
organization 

The  reduced  number  of  working 
groups  results  in  less  personal 
interaction 

High  rate  of  personnel  turnover 

Significant  turnover  in  JTB 
leadership 

Can’t  figure  out  who  to  talk  to  for 
certain  information 

Information  is  generally 
requested  from  unknown  points 
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of  contact 

People  don’t  know  who  to  ask  for 
what  information 

Some  individuals  don’t  know 
there  is  a  necessity  to  share 

Table  12  presents  the  organizational  structure  barriers  based  on 
information  collected  from  participant  surveys.  It  includes  participant  responses  related 
to  hierarchical  and  nonhierarchical  organizations. 


Table  12.  Organizational  Structure  Barriers 


Hierarchical 

Nonhierarchical 

Organizational  barriers  exist  between  services 

JTB  organization  is  constantly  changing 

Organizations  are  managed  as  "stovepipes" 

Geographical  distribution  of  JTB  organizations 
challenges  information  sharing 

The  JTB  does  not  have  sole  authority  over  is 
component  organizations,  which  results  in  a  lack  of 
unity  of  leadership  and  purpose 

Table  13  presents  the  motivational  enablers  based  on  infonnation 
collected  from  participant  surveys.  It  includes  participant  responses  related  to  intrinsic, 
intrinsic  hedonic,  and  extrinsic  motivation. 


Table  13.  Motivational  Enablers 


Intrinsic 

Intrinsic  Hedonic 

Extrinsic 

Sharing  information  to  improve 
everyone's  performance.  It 
doesn't  matter  who  takes  the 
credit,  as  long  as  the  job  is  done 
right 

Everything  is  done  for  the  war 
fighters,  and  they  are  appreciative 
of  it 

Information  required  for 
budgeting  is  received  quickly 

Share  to  clarify  information 

Everyone  has  emotional  ties  to 
making  sure  the  war  fighters  are 
safe 

Share  because  it  is  required  to  get 
the  job  done 

Individuals  will  use  information 
in  the  central  repository  correctly 
if  their  interest  is  to  help  the  guys 
overseas. 

Share  so  everyone  can  benefit 
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Table  14  presents  the  organizational  culture  enablers  based  on  information 
collected  from  participant  surveys.  It  includes  participant  responses  related  to  beliefs  and 
norms. 


Table  14.  Organizational  Culture  Enablers 


Beliefs  (Relating  to  Policy) 

Norms  (Relating  to  Process) 

Creating  standard  operating  procedures  within 
departments 

Multiple  ranges  cooperating  on  a  single  test 

Organizational  culture  based  on  reaction  to 
requirements  from  the  war  fighter  means  inherent 
flexibility 

Analysis  metrics  and  test  design  information  are 
often  shared  between  organizations 

Creating  an  environment  that  is  open  to  sharing  and 
protects  proprietary  interests 

Sharing  information  with  modeling  and  simulation 
may  lead  to  better  models  and  less  open  air  testing 

Guidance  from  higher  leadership 

Protocols  are  open  to  interpretation 

Internal  review  by  JIEDDO 

Some  commonality  of  requirements  between  test 
ranges 

The  new  deputy  director's  primary  goal  has  been  to 
improve  communication 

Working  Groups 

Most  people  get  the  amount  of  information  they 
need 

Weekly  status  reports  are  done  and  recorded 

Culture  is  better  now  than  it  was  two  or  three  years 
ago 

Table  15  presents  the  organizational  environment  enablers  based  on 
information  collected  from  participant  surveys.  It  includes  participant  responses  related 
to  personal  interaction,  information  systems,  and  incentives. 


Table  15.  Organizational  Environment  Enablers 


Personal  Interaction 

Information  Systems 

Incentive 

Personal  relationships  lead  to 
points  of  contact 

Transitioning  to  a  new  Oracle 
database 

Nothing  Noted 

Getting  information  depends  on 
personal  relationships 

The  JTB  is  creating  a  common 
server  environment  that  will 
attach  people  at  all  locations 

Personal  relationships  get  assets 

Establishing  a  threat  database  to 
track  threat  resources  and  their 

status 

Personal  relationships  get  access 
to  information 

JTB  portal  home  page  designed 
as  a  wheel  for  easy  access  to  all 
aspects  of  the  organization 
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Attending  conferences  to  develop 
personal  relationships  and 
establish  points  of  contact 

Read  Me  files  are  available  for 
everything  on  the  JTB  portal 

Working  groups  are  a  good  start 
to  communicating  between  ranges 

Access  controls  on  who  can  post 
to  the  JTB  portal  increase  trust  in 
the  posted  information 

Physical  collocation  of  leadership 
enables  better  communication 

Points  of  contact  for  the 

Knowledge  Integration  and 
Networking  Group  are  on  the  top 
of  the  page  of  the  JTB  Portal 

Weekly  meetings 

High  quality  hyperlinks  from  the 
JTB  portal 

Secure  Video  Teleconference 

Test  result  availability  on  the 
Theater  Support  Web  Tool 

Conferences 

Redundant  posting  on  the  Theater 
Support  Web  Tool  and  JTB  Portal 

Physical  collocation  of  testers 
from  multiple  ranges 

Tying  together  IT  infrastructure 

Face-to-face  groups  enhance 
sharing 

Theater  Support  Web  Tool 

Sharing  establishes  relationships 
that  make  it  easier  to  share  again 

Getting  together  to  solve 
problems 

JTB  portal 

Email  (NIPR  and  SIPR) 

Telephone 

Table  16  presents  the  organizational  culture  recommendations  based  on 
information  collected  from  participant  surveys.  It  includes  participant  responses  related 
to  beliefs  and  norms. 


Table  16.  Organizational  Culture  Recommendations 


Beliefs  (Relating  to  Policy) 

Norms  (Relating  to  Process) 

Change  the  culture  to  a  single  unified  team 

Restrict  access  of  information  to  individuals  who 
know  what  to  do  with  the  information  so  everyone 
is  willing  to  share 

JTB  should  provide  cleaner  roles  for  testing 
organizations 

Make  sure  everyone  is  aware  of  all  available 
resources 

Senior  Leadership  should  enact  a  new  management 
model  across  all  agencies,  at  all  levels  of  seniority 

Working  groups  should  do  a  better  job  of  letting 
others  know  what  information  they  have  to  share 

Create  a  position  to  consolidate  and  distribute 
information  and  updates  from  working  groups 

Continue  to  have  projects  where  different  test 
ranges  work  together 

Standardize  the  file  format  of  the  most  commonly 
used  test  data 

Create  procedures  for  information  to  flow  from  all 
locations  to  a  central  repository 

Formalize  the  information  sharing  processes 

Establish  the  processes  for  coordination 

Create  a  written  set  of  guidance  for  continuity  for 

Create  process  maps  for  data  flow  processes, 
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each  position  in  order  to  lessen  the  impact  of 
personnel  turnover 

infrastructure,  and  research  &  development 

Create  an  information  sharing  policy  for  the  JTB 

Data  being  collected  by  the  program  office  needs  to 
be  shared  with  the  JTB 

Develop  overarching  protocols 

Build  trust  with  JIEDDO  modeling  and  simulation 

Table  17  presents  the  organizational  environment  recommendations  based 
on  information  collected  from  participant  surveys.  It  includes  participant  responses 
related  to  personal  interaction,  information  systems,  and  incentives. 


Table  17.  Organizational  Environment  Recommendations 


Personal  Interaction 

Information  Systems 

Incentive 

Reinstate  JTB  run  range  worker 
meetings 

The  JTB  portal  should  include: 
climate  data,  atmospheric 
conditions,  time,  space,  and 
position  information 

Nothing  Noted 

Create  a  collective  situational 
awareness  for  the  organization 

A  central  repository  should 
include:  scenario  data, 
environmental  conditions, 
performance  data,  representation 
of  the  sensor  for  instrumentation, 
test  plans,  and  test  reports  that 
constitute  every  data  element  or 
data  record 

Get  the  JTB  leadership  together 
to  make  sure  that  everyone  is  on 
the  same  page 

Create  a  multi-security-level 
database  that  links  data  within 
reports  at  different  levels 

JTB  employees  should  visit  all  of 
the  organizations  to  understand 
how  they  work 

Add  a  selectable  pop-up  notice 
within  the  JTB  portal 

Conduct  an  offsite  to  give  people 
the  chance  to  get  to  know  each 
other 

An  easy  to  use  well  implemented 
web  interface 

Publish  a  complete  organizational 
chart  that  includes  individuals, 
their  expertise,  and  contact 
information 

Create  a  common  data  store 
between  the  JTB  and  JIEDDO 

JTB  needs  to  establish  points  of 
contact  at  each  test  location  that 
can  help  to  get  people  the  right 
test  data  that  they  need  to  be 
sharing 

Create  a  central  data  repository 
where  everyone  collects,  shares, 
and  presents  data  in  a  similar 
fashion 

Create  and  disseminate  a  point  of 
contact  list  for  each  organization 

All  data  in  a  central  repository 
should  be  appropriate  for  wide 
distribution 

Increase  telephone  discussions 
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2.  Affinity  Diagram 


a.  Background 

This  section  will  establish  the  focus  and  explain  the  presentation  of  the 
affinity  diagram.  As  stated  in  the  methodology  chapter,  the  primary  focus  for  this  affinity 
diagram  was  to  improve  information  sharing  within  the  JTB,  as  defined  for  this  thesis. 
With  that  in  mind,  the  secondary  integrated  and  supporting  focus  was  to  collect  data  that 
could  be  used  by  follow-on  researchers  in  the  creation  of  a  portal.  The  three  tiers 
represent  different  levels  of  abstraction  in  order  to  enable  the  user  to  focus  on  the  most 
pertinent  level. 

Due  to  the  quantity  of  data  points  in  Tier  III  of  the  diagram,  some 
explanation  of  how  to  read  the  tables  is  necessary.  First,  as  described  in  the  previous 
chapter,  the  diagram  is  built  from  the  bottom  up,  so  Tier  III  contains  all  of  the  qualitative 
data  points  from  all  interviews  and  surveys.  There  are  13  columns  in  Tier  III,  which  is 
presented  in  sections  for  readability.  As  a  result,  the  descriptions  before  each  Tier  III 
table  refer  to  column  numbers,  which  represent  the  original  organization  of  columns  from 
left  to  right.  Column  order  is  significant  because  the  left  to  right  organization  was 
correlated  between  all  three  tiers  of  the  diagram.  Figure  4  presents  a  shell  of  the  affinity 
diagram  in  order  to  clarify  how  the  sectioned  tables  of  Tier  III  incorporate  into  the 
diagram  as  a  whole. 
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Figure  4.  Shell  of  the  Affinity  Diagram 
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b.  Affinity  Diagram  Results 


The  results  of  the  affinity  diagram  are  presented  in  the  tables  that  follow. 
Table  18  presents  the  first  and  second  columns  of  Tier  III.  The  table  presents 
considerations  relevant  to  personal  relationships  and  individual  perceptions  in  the  design 
of  an  information  system  for  the  JTB. 


Table  18.  Tier  III  Diagram:  Personal  Relationships  and  Individual  Perceptions 


Personal  Relationships 

Individual  Perceptions 

The  corporate  culture  focuses  on  individuals  gaining 
power  over  others 

Individuals  don’t  see  themselves  as  part  of  an 
enterprise 

Sharing  is  often  contingent  upon  who  you  know 

Belief  throughout  the  organization  that  sharing 
information  may  result  in  losing  business  to  another 
organization 

Personal  relationships  lead  to  points  of  contact 

Maintain  organizational  reputation  by  the  provider 
restricting  sharing  to  only  those  who  he  believes 
understand  what  to  do  with  the  information 

Individuals  hoard  information  to  make  themselves 
seem  more  powerful 

Sharing  information  is  helping  your  competition, 
which  puts  your  job  at  risk 

Personal  relationships  get  assets 

Testers  and  range  workers  fear  losing  their  jobs 

Sharing  is  based  on  the  'good  ole  boy'  system 

Maintain  organizational  reputation  by  preventing 
misuse  of  data  by  other  organizations 

Personal  relationships  get  access  to  information 

Lack  of  trust  throughout  the  organization 

People  don’t  know  all  the  individuals  and 
personalities,  so  they  have  a  difficult  time  receiving 
the  information  they  need 

Individuals  don't  share  in  order  to  avoid  being 
judged  by  their  peers. 

Getting  information  depends  on  personal 
relationships 

Share  because  it  is  required  to  get  the  job  done 

Individual  biases 

Share  to  clarify  information 

Information  is  generally  requested  from  unknown 
points  of  contact 

The  belief  that  the  info  is  mine  so  no  one  else  can 
interpret  it 

Sometimes  an  ultimatum  is  necessary,  even  if  others 
know  the  information  is  required 

Individuals  will  use  information  in  the  central 
repository  correctly  if  their  interest  is  to  help  the 
guys  overseas. 

Some  individuals  share  only  what  is  requested, 
despite  knowing  more  is  required 

Sharing  information  to  improve  everyone's 
performance.  It  doesn't  matter  who  takes  the  credit, 
as  long  as  the  job  is  done  right 

Everyone  trying  to  secure  funding 

Share  so  everyone  can  benefit 

Attending  conferences  to  develop  personal 
relationships  and  establish  points  of  contact 

Individual's  fear  of  data  being  misused 

Conferences 

Everything  is  done  for  the  war  fighters,  and  they  are 
appreciative  of  it 

Working  group  networking 

Everyone  has  emotional  ties  to  making  sure  the  war 
fighters  are  safe 

Conduct  an  offsite  to  give  people  the  chance  to  get 
to  know  each  other 

Protect  the  war  fighter  by  ensuring  that  information 
is  not  taken  out  of  context 

Points  of  contact  for  the  Knowledge  Integration  and 
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Networking  Group  are  on  the  top  of  the  page  of  the 
JTB  Portal 

Sharing  establishes  relationships  that  make  it  easier 
to  share  again 

Table  19  presents  the  third  and  fourth  columns  of  Tier  III.  The  table 
presents  considerations  relevant  to  JTB  level  issues  and  leadership  in  the  design  of  an 
information  system  for  the  JTB. 


Table  19.  Tier  III  Diagram:  JTB  Level  Issues  and  Leadership 


JTB  Level  Issues 

Leadership 

Organizational  culture  based  on  reaction  to 
requirements  from  the  war  fighter  means  inherent 
flexibility 

The  corporate  culture  does  not  value  information 
sharing 

The  JTB  does  not  have  sole  authority  over  its 
component  organizations,  which  results  in  a  lack  of 
unity  of  leadership  and  purpose 

Culture  is  better  now  than  it  was  two  or  three  years 
ago 

Institutionalized  financial  resource  competition 
between  test  ranges  stops  information  sharing 

Change  the  culture  to  a  single  unified  team 

Physical  collocation  of  leadership  enables  better 
communication 

The  new  deputy  director's  primary  goal  has  been  to 
improve  communication 

Most  tests  are  set  up  as  individual  tests  to  be 
conducted  at  a  single  range. 

Senior  leadership  should  enact  a  new  management 
model  across  all  agencies,  at  all  levels  of  seniority 

Create  a  position  to  consolidate  and  distribute 
information  and  updates  from  working  groups 

Management  is  inadequate  at  multiple  levels 

Bureaucracy  has  made  the  organization  ineffective 

A  certain  test  range  will  share  only  complete  test 
reports  officially  approved  for  release 

Organizations  are  managed  as  "stovepipes" 

Managerial  resistance  to  obtaining  data  from 
another  organization 

JTB  organization  is  constantly  changing 

JTB  should  provide  cleaner  roles  for  testing 
organizations 

The  reduced  number  of  working  groups  results  in 
less  personal  interaction 

Requirements  continually  change 

Geographical  distribution  of  JTB  organizations 
challenges  information  sharing 

Requirements  levied  on  individuals  change 
frequently 

Management  intervention  is  often  necessary  to 
require  others  to  share  information 

Some  individuals  don’t  perceive  a  necessity  to  share 

Create  a  collective  situational  awareness  for  the 
organization 

Get  the  JTB  leadership  together  to  make  sure  that 
everyone  is  on  the  same  page 

Creating  an  environment  that  is  open  to  sharing  and 
protects  proprietary  interests 

Decisions  are  made  in  a  risk  averse  environment 

Guidance  from  higher  leadership 
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Table  20  presents  the  fifth  and  sixth  columns  of  Tier  III.  The  table 
presents  considerations  relevant  to  inter-organizational  communications  and  best 
practices  for  information  sharing  in  the  design  of  an  infonnation  system  for  the  JTB. 


Table  20.  Tier  III  Diagram:  Inter-Organizational  Communication  and  Best  Practices 


Inter-Organizational  Communication 

Best  Practices 

Reinstate  JTB  run  range  worker  meetings 

Continue  to  have  projects  where  different  test 
ranges  work  together 

Data  being  collected  by  the  program  office  needs  to 
be  shared  with  the  JTB 

Physical  collocation  of  testers  from  multiple  ranges 

Organizational  barriers  exist  between  services 

Multiple  ranges  cooperating  on  a  single  test 

Rivalries  between  sites  and  services  create  barriers 

Weekly  status  reports  are  done  and  recorded 

Sharing  information  with  modeling  and  simulation 
may  lead  to  better  models  and  less  open  air  testing 

Analysis  metrics  and  test  design  information  are 
often  shared  between  organizations 

Build  trust  with  JIEDDO  modeling  and  simulation 

Face-to-face  groups  enhance  sharing 

Internal  review  by  JIEDDO 

Getting  together  to  solve  problems 

JTB  employees  should  visit  all  of  the  organizations 
to  understand  how  they  work 

Working  groups  are  a  good  start  to  communicating 
between  ranges 

Test  ranges  have  different  priorities  than  the  JTB 
because  they  have  to  work  with  other  customers  to 
obtain  additional  financial  resources 

Weekly  SVTCs 

Increase  use  of  the  telephone  for  discussions 

Table  21  presents  the  seventh  and  eighth  columns  of  Tier  III.  The  table 
presents  considerations  relevant  to  standards  and  points  of  contact  in  the  design  of  an 
information  system  for  the  JTB. 


Table  21.  Tier  III  Diagram:  Standards  and  Points  of  Contact 


Standards 

Points  of  Contact 

No  commonality  of  a  strategic  plan  between  ranges 
or  from  higher  headquarters 

People  don’t  know  who  to  ask  for  what  information 

The  lack  of  a  written  Information  Sharing  policy 
results  in  confusion  through  reliance  on  a  set  of 
unwritten  norms  that  individuals  interpret 
differently 

Can’t  figure  out  who  to  talk  to  for  certain 
information 

Create  an  information  sharing  policy  for  the  JTB 

Publish  a  complete  organizational  chart  that 
includes  individuals,  their  expertise,  and  contact 
information 

Create  process  maps  for  data  flow  processes, 
infrastructure,  and  research  &  development 

Create  and  disseminate  a  point  of  contact  list  for 
each  organization 

Formalize  the  information  sharing  processes 

JTB  needs  to  establish  points  of  contact  at  each  test 
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location  that  can  help  to  get  people  the  right  test 
data  that  they  need  to  be  sharing 

Establish  the  processes  for  coordination 

There  is  no  process  in  place  to  share  information 

Create  procedures  for  information  to  flow  from  all 
locations  to  a  central  repository 

Creating  standard  operating  procedures  within 
departments 

There  are  no  formal  procedures  for  information 
sharing 

No  common  data  schema  or  data  collection  formats 

Develop  overarching  protocols 

Protocols  are  open  to  interpretation 

Some  commonality  of  requirements  between  test 
ranges 

There  is  no  standard  method  or  format  for 
requesting,  collecting,  storing,  or  disseminating  data 

Standardize  the  file  format  of  the  most  commonly 
used  test  data 

Table  22  presents  the  ninth  and  tenth  columns  of  Tier  III.  The  table 
presents  considerations  relevant  to  personnel  turnover  and  the  quality  of  shared 
information  in  the  design  of  an  infonnation  system  for  the  JTB. 


Table  22.  Tier  III  Diagram:  Personnel  Turnover  and  Quality  of  Shared  Information 


Personnel  Turnover 

Quality  of  Shared  Information 

High  rate  of  personnel  turnover 

Information  received  is  not  normally  timely  or 
accurate,  which  leads  to  getting  the  information 
through  different  means 

Significant  turnover  in  JTB  leadership 

When  sharing  occurs,  the  amount  of  information 
received  is  inadequate 

Create  a  written  set  of  guidance  for  continuity  for 
each  position  in  order  to  lessen  the  impact  of 
personnel  turnover 

Information  required  for  budgeting  is  received 
quickly 

Table  23  presents  the  eleventh  column  of  Tier  III.  The  column  was  split 
into  two  for  readability.  The  table  presents  considerations  relevant  to  information  sharing 
tools  in  the  design  of  an  information  system  for  the  JTB. 
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Table  23.  Tier  III  Diagram:  Information  Sharing  Tools 


Information  Sharing  Tools 

The  JTB  is  creating  a  common  server  environment 
that  will  attach  people  at  all  locations 

Create  a  common  data  store  between  the  JTB  and 
JIEDDO 

Tying  together  IT  infrastructure 

Create  a  multi-security-level  database  that  links  data 
within  reports  at  different  levels 

JTB  portal  home  page  designed  as  a  wheel  for  easy 
access  to  all  aspects  of  the  organization 

Create  a  central  data  repository  where  everyone 
collects,  shares,  and  presents  data  in  a  similar 
fashion 

Access  controls  on  who  can  post  to  the  JTB  portal 
increase  trust  in  the  posted  information 

A  central  repository  should  include:  scenario  data, 
environmental  conditions,  performance  data, 
representation  of  the  sensor  for  instrumentation,  test 
plans,  and  test  reports  that  constitute  every  data 
element  or  data  record 

High  quality  hyperlinks  from  the  JTB  portal 

The  enterprise  lacks  a  good  central  repository  that 
has  all  the  metadata  and  the  data  required  to  make 
use  of  the  information 

Read  Me  fdes  are  available  for  everything  on  the 

JTB  portal 

Establishing  a  threat  database  to  track  threat 
resources  and  their  status 

JTB  portal 

An  easy  to  use  well  implemented  web  interface 

The  JTB  portal  should  include:  climate  data, 
atmospheric  conditions,  time,  space,  and  position 
information 

Weekly  meetings 

Add  a  selectable  pop-up  notice  within  the  JTB 
portal 

Theater  Support  Web  Tool 

Theater  Support  Web  Tool  is  not  user  friendly 

Creation  of  a  database 

Transitioning  to  a  new  Oracle  database 

Secure  Video  Teleconference 

There  are  issues  with  permissions  in  the  information 
system 

Email  (NIPR  and  SIPR) 

Redundant  posting  on  the  Theater  Support  Web 

Tool  and  JTB  Portal 

SIPR 

Test  result  availability  on  the  Theater  Support  Web 
Tool 

Telephone 

Table  24  presents  the  twelfth  and  thirteenth  columns  of  Tier  III.  The  table 
presents  considerations  relevant  to  access  to  infonnation  and  information  visibility  in  the 
design  of  an  information  system  for  the  JTB. 


Table  24.  Tier  III  Diagram:  Access  to  Information  and  Information  Visibility 


Access  to  Information 

Information  Visibility 

Most  people  get  the  amount  of  information  they 
need 

Individuals  don’t  know  who  has  what  information 

People  get  the  information  they  need  only  after 
working  through  barriers 

Make  sure  everyone  is  aware  of  all  available 
resources 

There  is  a  lack  of  access  to  information  from  other 

Lack  of  visibility  on  updated  documents 
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facilities  and  ranges 

There  is  a  lack  of  access  to  documents  that  are 
known  to  be  available  to  others 

Falsely  assuming  that  others  have  access  to  the 
same  information 

Not  being  able  to  access  other  ranges  database  for 
information. 

Working  groups  should  do  a  better  job  of  letting 
others  know  what  information  they  have  to  share 

Obtaining  raw  unfiltered  data  from  a  test  range  is  a 
challenge 

Working  group  members  don’t  disseminate  what 
they  learn  with  co-workers 

Restrict  access  to  information  to  individuals  who 
know  what  to  do  with  the  information  so  everyone  is 
willing  to  share 

Inability  to  locate  documents  that  are  known  to 
exist  in  other  organizations 

People  are  reserved  in  large  video  teleconferences 
because  they  are  not  willing  to  share  with 
information  with  all  of  the  participants 

Meetings  that  include  coalition  partners  restrict  some 
communications 

Some  organizations  attempt  to  repress  certain 
information  if  it  might  be  harmful  to  a  system 

Much  of  the  information  gathered  is  proprietary  and 
not  shared 

Table  25  presents  the  complete  Tier  II  diagram.  The  table  presents  the 
information  at  a  higher  level  of  abstraction  by  categorizing  the  Tier  III  column  headings. 


Table  25.  Tier  II  Diagram:  Individual  Issues,  Leadership  Considerations, 
Management,  Information  Sharing,  and  Information  Availability 


Individual  Issues 

Leadership 

Considerations 

Management 

Information 

Sharing 

Information 

Availability 

Personal 

Relationships 

JTB  Level  Issues 

Best  Practices 

Quality  of  Shared 
Information 

Access  to 

Information 

Individual 

Perceptions 

Leadership 

Standards 

Information 

Sharing  Tools 

Information 

Visibility 

Inter- 

Organizational 

Communication 

Points  of  Contact 

Personnel 

Turnover 

Table  26  presents  the  complete  Tier  I  diagram.  The  table  presents  the 
information  at  the  highest  level  of  abstraction  by  categorizing  the  Tier  II  column 
headings. 
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Table  26.  Tier  I  Diagram:  Individuals,  Leadership,  and  Infonnation 


Individuals 

Leadership 

Information 

Individual  Issues 

Leadership  Considerations 

Information  Sharing 

Management 

Information  Availability 

C.  QUANTITATIVE  RESULTS 

1.  Consolidated  Information  Sharing  Survey  Results 

The  quantitative  data  from  the  initial  and  follow-up  infonnation  sharing  survey 
analysis  was  consolidated  and  run  through  the  descriptive  statistics  application  of  the  data 
analysis  add-on.  The  results  are  presented  in  Table  27.  The  program  returned  15 
statistics  for  each  question.  The  most  useful  statistics  for  this  analysis  were  the  mean, 
mode,  and  standard  deviation.  Presenting  all  of  the  survey  Likert  responses  for  each 
question  was  not  conducive  to  a  single  summary  table.  Therefore,  the  complete 
information  sharing  survey  text  and  Likert  responses  are  included  in  the  Appendix. 
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Table  27.  Consolidated  Infonnation  Sharing  Survey  Results 


# 

Question  Text 

Summary* 

N 

Mean 

Mode** 

Standard 

Deviation 

Organizational  Information  Sharing  Section 

1 

How  often  do  you  collaborate  or  share  information  with  other  organizations? 

Daily 

12 

1.33 

1 

0.78 

2 

The  main  reason  you  share  information  with  other  organizations  is: 

I  want  to 

12 

4 

1.24 

3 

In  your  experience  with  JTB  testing,  how  adequate  has  information  sharing  been  between  JTB  organizations? 

Adequate 

m 

3 

0.78 

m 

To  what  extent  do  organizations  provide  relevant  information  to  other  organizations  in  a  pro-active  way,  without  the  other  having  to  ask  for  it? 

Occasionally 

m 

3 

1.23 

5 

My  organization’s  mission  performance  depends  on  receiving  accurate  information  from  other  organizations. 

Somewhat  Agree 

12 

5 

1.83 

6 

My  organization  often  obtains  information  from  other  JTB  organizations  required  to  complete  the  mission. 

Somewhat  Agree 

12 

5 

1.92 

7 

My  organization  can  depend  on  information  from  other  organizations. 

Neutral 

12 

4 

1.56 

8 

Requests  for  information  outside  of  my  organization  are  answered  in  a  timely  manner. 

Often 

12 

4 

1.44 

m 

To  what  extent  has  information  you  received  from  other  organizations  facilitated  performance  of  your  tasks? 

Often 

12 

1.38 

EEl 

When  you  obtained  information  from  other  JTB  organizations  the  information  was:  Necessary 

Rarely 

12 

1.44 

m 

When  you  obtained  information  from  other  JTB  organizations  the  information  was:  Complete 

Rarely 

m 

3 

1.24 

EEl 

When  you  obtained  information  from  other  JTB  organizations  the  information  was:  Accurate 

Rarely 

12 

2.00 

0 

1.60 

EEl 

When  you  obtained  information  from  other  JTB  organizations  the  information  was:  Clear 

Rarely 

m 

2 

1.19 

10 

When  you  obtained  information  from  other  JTB  organizations  the  information  was:  Trusted 

Rarely 

m 

1.47 

10 

When  you  obtained  information  from  other  JTB  organizations  the  information  was:  Timely 

Rarely 

m 

3 

1.28 

11 

In  your  experience  with  the  JTB,  to  what  extent  are  you  able  to  get  the  information  you  need  from  other  organizations? 

12 

I  find  trying  to  share  information  with  other  organizations  and  the  JTB  unnecessarily  complex. 

m 

4 

1.34 

Intra-organization  Communication  Structure  Section 

1 

To  what  extent  do  you  observe  problems  caused  by  inadequate  communication  between  the  JTB  and  its  organizations? 

Occasionally 

12 

1.13 

2 

The  JTB  information  flow  process  allows  organizations  to  share  information  adequately  and  in  a  timely  manner. 

Neutral 

m 

4 

1.47 

3 

My  organization’s  role  in  the  testing  process  is  designed  in  such  a  way  that  we  need  to  interact  with  other  organizations  in  order  to  perform  our  mission  effectively. 

Somewhat  Agree 

12 

5 

1.99 

m 

My  organization’s  policies  encourage  sharing  information  with  other  organizations. 

m 

5 

1.73 

5 

The  information  flow  procedures  between  the  JTB  and  its  organizations  are  clearly  defined. 

Neutral 

12 

3.08 

4 

1.44 

Organizational  Culture  Section 

1 

My  organization  shares  ideas  openly  with  other  organizations. 

Somewhat  Agree 

12 

3.58 

5 

1.83 

2 

Other  organizations  can  only  successfully  complete  their  mission  tasks  if  they  receive  information  frommy  organization. 

Somewhat  Agree 

12 

4.08 

4 

1.38 

3 

Other  organizations  often  need  to  obtain  information  frommy  organization  to  complete  their  mission. 

Somewhat  Agree 

12 

4.17 

5 

1.40 

4 

My  organization  responds  to  requests  for  information  from  other  organizations  in  a  timely  manner. 

Often 

12 

4.00 

5 

1.54 

5 

The  information  gathered  at  my  organization  is  unique,  so  no  other  organization  would  find  it  useful. 

Somewhat  Disagree 

12 

1.67 

1 

1.37 

6 

Time  spent  sharing  and  reviewing  information  with  other  organizations  could  be  spent  more  efficiently. 

Neutral 

12 

3.08 

4 

1.51 

7 

How  often  is  information  from  current  tests  kept  within  your  organization  in  order  to  compete  for  future  test  contracts? 

Never 

12 

1.25 

0 

1.82 

8 

How  is  the  intent  of  keeping  this  information  for  future  competition  communicated? 

Explicitly 

12 

1.75 

0 

1.91 

*Summary  indicated  here  is  the  mean  rounded  to  the  nearest  whole  number  Likert  response. 

**0  represents  the  lack  of  a  response.  Likert  responses  that  are  represented  by  the  other  numbers  in  the  column  can  be  referenced  in  the  Appendix. 
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D.  CONCLUSION 


1.  Return  to  the  Purpose 

The  CTA  and  affinity  diagram  in  this  chapter  used  the  same  data  points  organized 
in  a  different  manner  for  different  purposes.  The  CTA  presented  reasons  why  people 
within  this  organization  made  the  cognitive  decision  whether  or  not  to  share  infonnation. 
The  affinity  diagram  presented  the  same  data  in  a  format  more  conducive  to  engineering 
a  computer  system  to  support  information  sharing  that  is  customized  for  this 
organization. 
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V.  CONCLUSIONS 


A.  OVERVIEW 

This  chapter  presents  conclusions  based  on  the  CTA  and  affinity  diagram  and 
provides  recommendations  for  improving  infonnation  sharing  within  the  JTB  and  future 
research. 

B.  CTA  CONCLUSIONS 

1.  Barriers  to  Information  Sharing 

Barriers  to  infonnation  sharing  were  recognized  in  the  categories  of  motivation, 
organizational  culture,  organizational  environment,  and  organizational  structure.  The 
type  of  motivation  most  often  recognized  for  causing  a  barrier  was  intrinsic  hedonic 
motivation.  Individuals  were  motivated  to  not  share  their  information  in  order  to  make 
them  feel  better.  The  feelings  were  often  in  regard  to  maintaining  a  reputation,  attaining 
power  over  others,  or  protecting  the  purity  of  their  infonnation. 

Organizational  culture  barriers  were  individually  and  collectively  the  most 
common  data  points  out  of  all  categories  in  the  CTA.  While  the  number  of  data  points  is 
not  necessarily  as  relevant  as  their  substance,  it  is  significant  that  this  category  received 
so  much  attention  by  the  participants.  In  the  creation  of  the  CTA  tables,  beliefs  regarding 
sharing  information  related  to  policy  or  higher  headquarters  issues  and  norms  were 
process  and  procedural  issues.  Individuals  did  not  share  information  because  the 
organization  did  not  have  the  formalized  policy,  process,  or  procedures  in  place  to  create 
a  culture  that  encouraged  sharing. 

Participants  recognized  factors  related  to  personal  interaction  as  the  most 
significant  cause  of  barriers  regarding  the  organizational  environment.  Few  data  points 
related  to  information  systems,  and  none  related  to  incentives.  Personal  interaction  issues 
centered  on  the  participants  not  sharing  information  or  not  being  able  to  access 

information  due  to  the  lack  of  personal  points  of  contact  in  other  parts  of  the 

59 


organization.  It  is  interesting  to  note  that  there  is  the  same  number  of  data  points 
recognizing  personal  interaction  as  an  enabler  to  information  sharing,  but  for  different 
reasons,  which  will  be  discussed  in  that  section.  Noting  relatively  few  data  points  that 
categorize  information  systems  as  a  barrier  to  sharing  information  is  also  significant 
when  considering  what  aspects  of  the  organization  to  fix  or  enhance  to  improve 
information  sharing.  The  lack  of  data  points  related  to  incentives  will  be  addressed  in  the 
future  research  section. 

The  nonhierarchical  aspect  of  organizational  structure  contained  more  data  points 
than  the  hierarchical  aspect.  Issues  centered  on  the  dynamic  and  dispersed  structure  and 
the  JTB’s  lack  of  authority  to  control  its  suborganizations,  which  results  in  each 
suborganization  being  managed  as  a  separate  “stovepipe.” 

2.  Enablers  for  Information  Sharing 

Enablers  for  information  sharing  were  recognized  in  the  categories  of  motivation, 
organizational  culture,  and  organizational  environment.  No  enablers  were  recognized 
relative  to  organizational  structure.  Intrinsic  and  intrinsic  hedonic  motivations  were 
essentially  equal  influences  for  enabling  infonnation  sharing.  Participants  made  the 
decision  to  share  information  because  they  felt  it  was  the  right  thing  to  do  and/or  because 
it  made  them  feel  good  that  they  were  helping  the  war  fighters. 

Both  nonns  and  beliefs  were  recognized  as  organizational  culture  enablers. 
Norms  related  to  cooperation  between  ranges  as  well  as  between  ranges  and 
organizations  that  conduct  modeling  and  simulation.  Beliefs  recognized  an  increasing 
leadership  focus  on  infonnation  sharing. 

Organizational  environment  enablers  were  related  to  infonnation  systems  and 

personal  interaction.  As  with  the  barrier  section,  no  data  points  were  noted  under 

incentives.  Information  systems  data  points  mostly  related  to  the  JTB  portal  and  provide 

a  good  source  of  feedback  for  developers  and  considerations  for  future  improvements. 

Participants  also  recognized  the  importance  of  e-mail  and  the  telephone,  which  ties  in  the 

personal  interaction  aspect.  While  personal  interaction  was  recognized  as  a  barrier  due  to 

personal  points  of  contact,  once  those  points  of  contact  are  made  they  are  recognized  as 
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enablers.  Another  aspect  of  personal  interaction  recognized  as  an  enabler  was  the 
opportunity  to  communicate  face  to  face,  whether  to  create  points  of  contact  or  exchange 
information  between  existing  relationships. 

3.  Participant  Recommendations  for  Improvement 

Participant  recommendations  for  improving  infonnation  sharing  were  recognized 
in  the  categories  of  organizational  culture  and  organizational  environment.  No 
recommendations  were  noted  regarding  motivation  or  organizational  structure.  Both 
beliefs  and  norms  were  recognized  by  organizational  culture  recommendations. 
Recommendations  for  beliefs  centered  on  improving  leadership  and  increasing 
standardization.  Norms  mostly  dealt  with  different  aspects  of  dissemination,  to  include 
improving  processes,  procedures,  and  visibility. 

Organizational  environment  recommendations  were  shared  between  personal 
interaction  and  information  systems;  again,  incentives  were  not  noted.  Personal 
interaction  recommendations  focused  on  meeting  in  person  and  dissemination  of  a 
complete  organizational  chart  with  amplifying  information. 

Information  system  recommendations  included  some  specific  types  of  information 
to  be  included  in  a  central  repository  of  information,  as  well  as  improvements  for  the 
current  JTB  portal. 

C.  AFFINITY  DIAGRAM  CONCLUSIONS 

1.  Levels  of  Abstraction 

The  information  contained  in  the  affinity  diagram  provides  a  tool  for  planners  and 
designers  to  use  if  JTB  leadership  decides  that  designing  an  information  system  is 
appropriate  and  feasible  to  improve  information  sharing.  It  focuses  on  feedback  to 
planners  and  designers  rather  than  understanding  why  members  of  the  organization 
decide  whether  or  not  to  share.  As  such,  it  is  a  good  companion  method  to  the  CTA,  but 
leads  to  less  telling  conclusions.  As  a  tool  for  planners  and  designers,  the  three  tiers  of 
the  affinity  diagram  essentially  represent  three  levels  of  abstraction  of  the  data  that  can  be 

61 


used  to  customize  the  design  of  an  information  system  to  improve  infonnation  sharing 
within  the  JTB.  Tier  I,  the  highest  level  of  abstraction,  identifies  the  issues  that  need  to 
be  addressed.  Tier  II,  the  middle  level,  guides  planners  in  deciding  what  areas  to  focus 
on  when  developing  requirements,  an  aspect  of  great  importance  prior  to  development. 
Tier  III  gets  into  the  weeds  and  provides  details  and  recommendations  useful  to  planners 
and  designers.  It  must  be  recognized,  however,  that  not  all  data  points  included  in  Tier 
III  are  feasible  or  within  the  control  of  the  organization. 

2.  Affinity  Diagram  Conclusions 

Tier  II  is  the  most  appropriate  tier  to  examine  when  considering  conclusions.  It 
identifies  individual  issues,  leadership  considerations,  management,  information  sharing, 
and  information  availability  as  focus  areas.  When  considering  how  designing  an 
information  system  can  improve  information  sharing,  leaders  and  planners  must  home  in 
on  what  aspects  technology  can  feasibly  address.  For  example,  it  would  not  be  feasible 
to  focus  the  time,  effort,  and  resources  necessary  to  directly  target  individual  perception. 
However,  by  targeting  other  focus  areas,  individual  perception  may  be  influenced. 
Narrowing  the  focus  to  targeting  relevant  data  points  under  management,  information 
sharing,  and  infonnation  availability  would  be  a  good  starting  point.  Planners  should 
also  consider  feasible  data  points  from  other  categories  and  how  changes  in  the  targeted 
data  points  may  also  indirectly  address  other  categories  and  data  points.  Once  a  plan  is 
being  created,  pulling  data  points  from  Tier  III  will  provide  amplification  and  clarity. 

D.  INFORMATION  SHARING  SURVEY  CONCLUSIONS 

1,  Organizational  Information  Sharing 

The  organizational  information  sharing  section  results  presented  the  level  of 
information  sharing  efficiency  between  organizations.  Although  the  average  participant 
collaborated  daily  and  the  received  information  often  facilitated  task  performance,  the 
information  was  rarely  complete,  accurate,  clear,  or  trusted. 
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2. 


Intra-Organization  Communication  Structure 


The  intra-organizational  communication  structure  section  results  presented  each 
participants’  evaluation  of  how  well  information  flows  within  their  organization.  The 
majority  of  participants  perceived  the  need  to  interact  and  responded  that  their 
organization’s  policies  encouraged  information  sharing. 

3.  Organizational  Culture 

The  organizational  culture  section  results  presented  each  participant’s  evaluation 
of  the  organizational  culture.  The  majority  perceived  the  necessity  of  giving  and 
receiving  information  and  responded  that  theirs  shares  openly. 

E.  RECOMMENDATIONS 

1.  Overall 

CTA  provides  focus  areas  for  leadership  to  address  when  considering  how  to 
increase  the  likelihood  of  members  sharing  information.  The  information  sharing  survey 
enables  the  participants  to  evaluate  multiple  aspects  of  infonnation  sharing  between  and 
within  organizations.  The  affinity  diagram  is  a  tool  that  provides  a  starting  point  for 
designing  an  infonnation  system  that  enables  the  members  to  share  information  by 
enhancing  what  they  have  identified  as  the  strengths  and  improving  what  they  have 
identified  as  the  weaknesses. 

2.  CTA  Recommendations 

This  section  presents  the  author’s  recommendations  to  address  the  most  common 
barriers  to  and  enablers  of  information  sharing  within  the  JTB.  User  recommendations 
presented  earlier  in  this  chapter  should  be  considered  when  implementing  and  changes 
relative  to  information  sharing. 

The  most  common  barriers  were  organizational  culture,  both  beliefs  and  norms, 
and  personal  interaction.  In  order  to  mitigate  those  barriers,  the  researcher  recommends 
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that  the  JTB  create  a  position  to  focus  specifically  on  infonnation  flow  both  within  and 
outside  of  the  organization.  The  position  would  initially  be  concerned  with  improving 
information  sharing  by  recognizing  and  mitigating  barriers  by  being  a  ‘go  to’  person  for 
information  dissemination  and  access  issues. 

Areas  recognized  as  enablers  that  should  be  enhanced  include  the  organizational 
culture  norm  of  cooperation  between  test  ranges  and  the  information  systems  aspect  of 
the  organizational  environment.  To  enhance  cooperation  between  ranges,  this  researcher 
recommends  increasing  meetings  between  testers  from  different  ranges.  Including  the 
actual  testers  is  significant  because  they  are  the  people  who  generate  the  information  that 
everyone  else  will  share.  If  they  create  a  culture  that  includes  sharing,  it  will  influence 
the  motivation  of  others  and  the  norms  of  the  organization.  Also,  the  positive  image  of 
information  systems  by  the  participants  leads  to  the  recommendation  to  enhance  the 
capability  of  information  systems  to  share  information.  Investing  in  JTB  portal 
improvement  and  including  some  of  the  user  recommendations  would  increase  sharing 
capability  and  let  the  members  of  the  organization  know  that  their  leadership  values  their 
input. 


3.  Affinity  Diagram 

Infonnation  sharing  within  the  JTB  would  improve  if  JTB  leadership  provided  a 
system  that  could  enhance  building  personal  relationships  and  standardize  the 
information  sharing  process  in  order  to  improve  its  quality  and  timeliness,  while 
improving  information  visibility  and  access.  Whether  or  not  that  is  feasible  is  up  to  the 
JTB  leadership.  A  recommendation  more  applicable  at  this  point  in  the  evolving  status  of 
the  JTB  is  that  any  improvements  to  information  systems  within  the  JTB  should  at  least 
include  a  review  of  the  affinity  diagram  because  it  is  customized  for  the  JTB.  Other 
organizations  can  reference  the  diagram  as  an  example  of  how  to  customize  their 
information  technology  strengths  and  weaknesses. 
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4. 


Information  Sharing  Survey 


Results  from  the  information  sharing  survey  reinforce  the  recommendation  to 
create  a  position  to  focus  specifically  on  information  flow  both  within  and  outside  of  the 
organization.  Individuals  collaborate  daily,  perceive  the  need,  and  perceive  infonnation 
sharing  positively.  Having  an  individual  focus  on  making  the  information  flow  more 
efficient  would  improve  both  organizational  culture  and  performance. 

5.  Future  Research 

Future  research  on  incentivizing  infonnation  sharing  for  JTB  members  may  prove 
useful.  None  of  the  participants  provided  data  points  that  related  to  incentives  and  there 
were  few  data  points  related  to  extrinsic  motivation,  but  that  was  likely  due  to  the  focus 
of  this  research  being  to  identify  baniers  and  mitigate  them.  Because  there  is  no 
incentive  system,  it  is  not  a  banier,  but  follow-on  research  could  focus  on  incentive 
systems  for  government  employees  and  contractors.  While  intrinsic  and  intrinsic  hedonic 
motivations  are  powerful,  there  is  always  a  point  where  some  extrinsic  motivation  can 
make  the  difference  between  deciding  whether  or  not  to  share. 
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APPENDIX.  INFORMATION  SHARING  SURVEY  TEXT 


The  text  of  the  information  sharing  survey  distributed  to  all  participants  is  presented  below.  It 
provides  clarification  on  the  statistical  mean  column  of  the  Consolidated  Information  Sharing 
Survey  Results  table  presented  in  Chapter  IV. 

Organizational  Information  Sharing 


1 .  How  often  do  you  collaborate  or  share  information  with  other  organizations? 


0 

1 

2 

3 

4 

No  Response 

Daily 

Weekly 

Monthly 

Quarterly 

2.  The  main  reason  you  share  infonnation  with  other  organizations  is: 


0 

1 

2 

3 

4 

No  Response 

I  don’t 

I  was  directed  to 

Everyone  else 
does 

I  want  to 

3.  In  your  experience  with  JTB  testing,  how  adequate  has  information  sharing  been 
between  JTB  organizations? 


0 

1 

2 

3 

4 

5 

No 

Response 

No 

information 
has  been 
shared 

Too  much 
information 
sharing 

Inadequate 

Adequate 

Excellent 

4.  To  what  extent  do  organizations  provide  relevant  information  to  other  organizations  in 
a  pro-active  way,  without  the  other  having  to  ask  for  it? 


0 

1 

2 

3 

4 

5 

No 

Response 

Never 

Rarely 

Occasionally 

Often 

Always 

5.  My  organization’s  mission  performance  depends  on  receiving  accurate  infonnation 
from  other  organizations. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

disagree 

Somewhat 

disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 

69 


6.  My  organization  often  obtains  information  from  other  JTB  organizations  required  to 
complete  the  mission. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

disagree 

Somewhat 

disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 

7.  My  organization  can  depend  on  information  from  other  organizations. 


0 

1 

2 

3 

4 

5 

No  Response 

Strongly 

disagree 

Somewhat 

disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 

8.  Requests  for  information  outside  of  my  organization  are  answered  in  a  timely  manner. 


0 

1 

2 

3 

4 

5 

No 

Response 

Never 

Rarely 

Occasionally 

Often 

Always 

9.  To  what  extent  has  information  you  received  from  other  organizations  facilitated 
performance  of  your  tasks? 


0 

1 

2 

3 

4 

5 

No 

Response 

Never 

Rarely 

Occasionally 

Often 

Always 

10.  When  you  obtained  information  from  other  JTB  organizations  the  information  was: 
Necessary: 


0 

1 

2 

3 

4 

No  Response 

Never 

Rarely 

Always 

N/A 

Complete: 


0 

1 

2 

3 

4 

No  Response 

Never 

Rarely 

Always 

N/A 

Accurate: 


0 

1 

2 

3 

4 

No  Response 

Never 

Rarely 

Always 

N/A 
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Clear: 


0 

1 

2 

3 

4 

No  Response 

Never 

Rarely 

Always 

N/A 

Trusted: 


0 

1 

2 

3 

4 

No  Response 

Never 

Rarely 

Always 

N/A 

Timely: 


0 

1 

2 

3 

4 

No  Response 

Never 

Rarely 

Always 

N/A 

1 1 .  In  your  experience  with  the  JTB,  to  what  extent  are  you  able  to  get  the  information 
you  need  from  other  organizations?  Results  were  integrated  with  other  qualitative  data 
from  the  initial  round  of  data  collection. 

12.  I  find  trying  to  share  information  with  other  organizations  and  the  JTB  unnecessarily 
complex. 


0 

1 

2 

3 

4 

5 

No  Response 

Never 

Rarely 

Occasionally 

Often 

Always 

Intra-organization  Communications  Structure 


1.  To  what  extent  do  you  observe  problems  caused  by  inadequate  communication 
between  the  JTB  and  its  organizations? 


0 

1 

2 

3 

4 

5 

No  Response 

Never 

Rarely 

Occasionally 

Often 

Always 

2.  The  JTB  information  flow  process  allows  organizations  to  share  information 
adequately  and  in  a  timely  manner. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 
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3.  My  organization’s  role  in  the  testing  process  is  designed  in  such  a  way  that  we  need  to 
interact  with  other  organizations  in  order  to  perform  our  mission  effectively. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 

4.  My  organization’s  policies  encourage  sharing  information  with  other  organizations. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 

Somewhat 

Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 

5.  The  infonnation  flow  procedures  between  the  JTB  and  its  organizations  are  clearly 
defined. 


0 

1 

2 

3 

4 

5 

No 

Response 

Strongly 

Disagree 
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Disagree 
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Somewhat 

Agree 

Strongly 

Agree 

Organizational  Culture 

1 .  My  organization  shares  ideas  openly  with  other  organizations. 
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No 

Response 

Strongly 

Disagree 
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Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 

2.  Other  organizations  can  only  successfully  complete  their 
information  from  my  organization. 

mission  tasks  if  they  receive 
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Strongly 

Disagree 
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3.  Other  organizations  often  need  to  obtain  infonnation  from  my  organization  to 
complete  their  mission. 
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Strongly 

Disagree 
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Disagree 
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Agree 

Strongly 

Agree 
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4.  My  organization  responds  to  requests  for  information  from  other  organizations  in  a 
timely  manner. 
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Response 
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Often 

Always 

5.  The  information  gathered  at  my  organization  is  unique,  so  no  other  organization  would 
find  it  useful. 
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Strongly 

Disagree 
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Agree 

6.  Time  spent  sharing  and  reviewing  information  with  other  organizations  could  be  spent 
more  efficiently. 
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Disagree 

Neutral 

Somewhat 

Agree 

Strongly 

Agree 

7.  How  often  is  information  from  current  tests  kept  within  your  organization  in  order  to 
compete  for  future  test  contracts? 
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No  Response 

Never 
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Often 

Always 

8.  How  is  the  intent  of  keeping  this  infonnation  for  future  competition  communicated? 
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3 

4 

No 

Response 

Implicitly 

Explicitly 

Both 

No  infonnation  is  kept  for 
future  competition 
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INITIAL  DISTRIBUTION  LIST 


1.  Defense  Technical  Information  Center 
Fort  Belvoir,  Virginia 

2.  Dudley  Knox  Library 
Naval  Postgraduate  School 
Monterey,  California 

3.  Marine  Corps  Representative 
Naval  Postgraduate  School 
Monterey,  California 

4.  Director,  Training  and  Education,  MCCDC,  Code  C46 
Quantico,  Virginia 

5.  Director,  Marine  Corps  Research  Center,  MCCDC,  Code  C40RC 
Quantico,  Virginia 

6.  Marine  Corps  Tactical  Systems  Support  Activity  (Attn:  Operations  Officer) 
Camp  Pendleton,  California 

7.  Joint  Improvised  Explosive  Device  Defeat  Test  Board 

Alexandria,  Virginia 

8.  Department  Chair,  Information  Sciences  Department 

Naval  Postgraduate  School 

Monterey,  California 
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